minus-squarepewter@lemmy.worldtoTechnology@lemmy.world•AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.linkfedilinkEnglisharrow-up2·8 days agoThere’s a chance this isn’t being done by someone who only wants Wikipedia’s data. As the amount of websites you scrape increases, your desire to use the easy tools loses out to creating the most general tool that can look at most webpages. linkfedilink
minus-squarepewter@lemmy.worldtoFediverse@lemmy.world•So, after using Lemmy for 1.5 Years. You are telling I am not even allowed to use Lemmy?linkfedilinkEnglisharrow-up6·1 month agoI like how knowing all these lyrics will convince people that all of you are older than 23. linkfedilink
There’s a chance this isn’t being done by someone who only wants Wikipedia’s data. As the amount of websites you scrape increases, your desire to use the easy tools loses out to creating the most general tool that can look at most webpages.