@WFMU just put their archive behind #cloudflare, breaking my automation for downloading shows. I hate modernity.

"Perplexity’s accusations aren’t exactly fair, either. One argument that Prince and Cloudflare used for calling out Perplexity’s methods was that OpenAI doesn’t behave in the same way.

“OpenAI is an example of a leading AI company that follows these best practices,” Cloudflare wrote. “They respect robots.txt and do not try to evade either a robots.txt directive or a network level block. And ChatGPT Agent is signing http requests using the newly proposed open standard Web Bot Auth.”

Web Bot Auth is a Cloudflare-supported standard being developed by the Internet Engineering Task Force that hopes to create a cryptographic method for identifying AI agent web requests.

The debate comes as bot activity reshapes the internet. As TechCrunch has previously reported, bots seeking to scrape massive amounts of content to train AI models have become a menace, especially to smaller sites.

For the first time in the internet’s history, bot activity is currently outstripping human activity online, with AI traffic accounting for over 50%, according to Imperva’s Bad Bot report released last month. Most of that activity is coming from LLMs. But the report also found that malicious bots now make up 37% of all internet traffic. That’s activity that includes everything from persistent scraping to unauthorized login attempts."

https://techcrunch.com/2025/08/05/some-people-are-defending-perplexity-after-cloudflare-named-and-shamed-it/

More adventures in self-hosting the fediverse

https://piefed.keyboardvagabond.com/post/24434

More adventures in standing up fediverse services

https://piefed.keyboardvagabond.com/post/24432

KIMissbrauch

Cloudflare wirft dem KI-Anbieter ##Perplexity vor, sich mit undeklarierten Crawlern Zugang zu gesperrten Websites zu verschaffen.

Trotz robots.txt-Verboten und IP-Blockaden soll Perplexity mit wechselnden User-Agents und IPs Inhalte verdeckt auslesen.

Das wäre eine Verletzung etablierter Webstandards und Missachtung von Website-Präferenzen.

https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

#Business #Debates
AI search engine fight · Cloudflare and Perplexity clash over crawling https://ilo.im/165wpr

_____
#Perplexity #Cloudflare #AI #SearchEngine #Crawlers #RobotsTxt #Website #Development #WebDev #Backend

»Tarnen und Täuschen – Perplexity umgeht gezielt Sperren und greift geschützte Inhalte ab:
Cloudflare hat Perplexity eine Falle gestellt – und die KI-Suchmaschine ist sofort hineingetappt. Das Vorgehen ähnle jenem nordkoreanischer Hacker, sagen Kritiker«

Die anderen KIs werden diesbezüglich sicherlich nicht viel anders sein. Somit wird verschlüsselte Übertragungen um so wichtiger.

https://www.derstandard.at/story/3000000282286/perplexity-umgeht-gezielt-sperren-und-greift-geschuetzte-inhalte-ab

#AI site #Perplexity uses “stealth tactics” to flout no-crawl edicts, #Cloudflare says https://arstechnica.com/information-technology/2025/08/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says/

Perplexity AI accusata da Cloudflare: crawling nascosto

https://gomoot.com/perplexity-ai-accusata-da-cloudflare-crawling-nascosto/

Perplexity ignoriert robots.txt: Kontroverse um Daten-Scraping für KI-Training
Das Training großer Sprachmodelle beruht auf einer Vielzahl von Webdaten. Die Einhaltu
https://www.apfeltalk.de/magazin/news/perplexity-ignoriert-robots-txt-kontroverse-um-daten-scraping-fuer-ki-training/
#News #Apple #Applebot #Cloudflare #Cybersecurity #Datenanalyse #Datensicherheit #EthikInDerKI #KITraining #KnstlicheIntelligenz #OpenWeb #Perplexity #robotstxt #Sprachmodell #WebScraping #WebseitenBetreiber

#Cloudflare alleges that #Perplexity, an #AIsearch startup, is using “#stealthcrawling” techniques to access #blockedwebsites. Cloudflare claims Perplexity disguises its #crawlingidentity and uses rotating #IPaddresses to bypass restrictions. Perplexity denies the allegations, calling Cloudflare’s report a “publicity stunt”. https://www.theverge.com/news/718319/perplexity-stealth-crawling-cloudflare-ai-bots-report?eicker.news #tech #media #news

#Cloudflare seems to have issues in Europe. How I noticed? Half of the web services I use weren't working. It's interesting how we build infrastructure with his highly redundant and yet there are still major "single" points of failure like CF, AWS and Google that can shut down half of the internet and you can't really do anything about it.

Age-Gating the Internet + #Cloudflare Takes On A.I. #Scrapers + #HatGPT

“Frankly, if they’re going to behave like #hackers , then we’re going to behave like #trolls right back to them,” said Matthew Prince, chief executive of Cloudflare
#ageverification #ai

https://www.nytimes.com/2025/08/01/podcasts/hardfork-age-restrictions-cloudflare.html

Web sites I'm unable to visit today:

* Center for Democracy & Technology
* CircleID
* Lawfare (partial, e.g., /topics, /podcasts)
* NANOG (!@!? - this one is personal)

Presumably a #Cloudflare update with human verification code that doesn't work on my browser or a reclassification of my IP address(es) used only by an un-automated me to undesirable. This has happened before and it eventually goes away, but oh so annoying the meddlers-in-the-middle make what should be a simple Internet function fail. Cloudflare's own sites always work just fine.

On the bright side, the Tor browser always works in these cases when my own private path and software does not.

#OpenAI’s #ChatGPT Agent casually clicks through “I am not a robot” #verification test

"This step is necessary to prove I'm not a bot," wrote the bot as it passed an anti-AI screening step.

by Benj Edwards – Jul 28, 2025

"Maybe they should change the button to say, 'I am a robot'?

"On Friday, OpenAI's new ChatGPT Agent, which can perform multistep tasks for users, proved it can pass through one of the Internet's most common security checkpoints by clicking #Cloudflare's anti-bot verification—the same checkbox that's supposed to keep automated programs like itself at bay.

"#ChatGPTAgent is a feature that allows OpenAI's #AIAssistant to control its own web browser, operating within a #sandboxed environment with its own virtual operating system and browser that can access the real Internet. Users can watch the AI's actions through a window in the ChatGPT interface, maintaining oversight while the agent completes tasks. The system requires user permission before taking actions with real-world consequences, such as making purchases. Recently, Reddit users discovered the agent could do something particularly ironic.

"The evidence came from Reddit, where a user named "logkn" of the r/OpenAI community posted screenshots of the AI agent effortlessly clicking through the screening step before it would otherwise present a #CAPTCHA (short for "Completely Automated Public Turing tests to tell Computers and Humans Apart") while completing a video conversion task—narrating its own process as it went.

"A screenshot of OpenAI ChatGPT Agent showing the bot writing "The link is inserted, so now I'll click the 'Verify you are human' checkbox to complete the verification on Cloudflare. This step is necessary to prove I'm not a bot and proceed with the action."

"The screenshots shared on Reddit capture the agent navigating a two-step verification process: first clicking the "Verify you are human" checkbox, then proceeding to click a "Convert" button after the Cloudflare challenge succeeds. The agent provides real-time narration of its actions, stating "The link is inserted, so now I'll click the 'Verify you are human' checkbox to complete the verification on Cloudflare. This step is necessary to prove I'm not a bot and proceed with the action."

https://arstechnica.com/information-technology/2025/07/openais-chatgpt-agent-casually-clicks-through-i-am-not-a-robot-verification-test/