Jesus it’s not even like it just ran rmdir from the current directory and got lost, it just flat out explicitly deleted everything from the root of the drive

Yeah as an admin I love that I can run familiar Linuxy commands in powershell but I also hate that they can’t just use/fork the real userland utilities so everything works just similarly enough to completely throw you off when you stumble across a difference

I love reading this as the backups have never succeeded

Its all creepy until you realize it was all just a chat with an LLM and not actually an agentic machine learning model or chain of models hooked into some custom APIs

LLMs famously collapse into rediculousness once a conversation goes on too long. They’re now at the point where that takes more than a couple of paragraphs of text at least

On the upside, they specified D: drive which is typically a lesser used bulk storage drive, so less activity to potentially overwrite the files marked as deleted

Hey it’s never too late to get things better under control! My parents only just started that journey and they’re old enough to be grandparents

Honestly the questions you’re posing require a level of market analysis that could fill an entire white paper and be sold for way more money than I want to think about. Its a level of market analysis I don’t want to dive into. My gut instinct from having worked in the tech industry, working with datacenters and datacenter hardware at large companies is that the AI industry will contract significantly when the bubble pops. I’m sure I could find real data to support this prediction but the level of analysis that would require and the hours of work are simply more than it’s worth for an internet comment.

You have factors including what hardware is being deployed to meet AI bubble demand, how the networking might be setup differently for AI compared to general GPU compute, who is deploying what hardware, what the baseline demand for GPU compute is if you simulate no present AI bubble, etc. etc. it’s super neat data analysis but I ain’t got the time nor appetite for that right now

That’s super interesting! I didn’t know other languages handled side thoughts better!

Oh yeah machine learning as a technology will survive, and eventually it will be implemented where it can do what it’s really good at, but right now it’s being shoved into everything to do things it isn’t good at, so you end up with a super expensive to run, energy inefficient tool that runs worse than with traditional algorithms that can be run client side or on a single much cheaper server (I’m oversimplifying the server architecture for brevity)

Think customer service chatbots on ever car dealership’s website. Traditionally these were extremely simplistic and usually just had canned responses based on keywords in the customer’s written message and would quickly cascade the customer to a real customer service rep as soon as things got out of scope. Now with LLMs companies are running those as the customer service chatbots and the LLM can do anything from agreeing to sell a new car for a dollar to providing scam or invalid contact info to referring the customer to a competitor. There’s no knowing what the AI will do because it’s non-deterministic and you don’t want that in customer service!

Right now we’re in the bubble phase where every single company is finding some way to shoehorn AI into its business model so they can brag about it. Fucking Logitech added a remappable AI button that brings up a ChatGPT interface and just spends Logitech’s money on tokens with ChatGPT. That’s pure bubble behavior. Once the bubble pops we won’t have literally every single time you open a car dealership page spending an LLM token or 5, you won’t have Amazon running AI chatbots on every product page just for asking about that product, you won’t have every website just giving away free unrestricted access to LLMs. That’s what I’m talking about.

AI demand will drop when the bubble pops, and while it will be higher than it was 8 years ago, everyone is going to be very skeptical of anything AI, just like folks are still skeptical of mortgage backed securities over 15 years later, or just like people are skeptical of commerical websites without a clear method of financing 25 years after the dotcom bubble. People remember these things and will take a while to warm up to the idea again

Is there enough demand for thousands of servers with purpose built ARM processors (which may or may not have any publicly available kernel support) driving 4-8 600w a pop Nvidia datacenter chips though? Yes some will be repurposed but there simply won’t be the demand to fill immediately. Realistically what will happen is companies operating these datacenters will liquidate the racks, probably liquidate some of the datacenters entirely and thousands of servers will hit the secondhand market for next to nothing. While some datacenter structure city empty and unmaintained until they’re either bought up to be repurposed, bought up to be refurbished and brought back into datacenter use of torn down, just like an empty Super Walmart location

Some of the datacenters will be reworked for general compute, maybe a couple will maintain some AI capacity, but given the sheer quantity of compute being stood up for the AI bubble and the sheer scale of the bubble, basically every major tech company is likely to shrink significantly when the bubble pops, since we’re talking companies that currently have market caps measured in trillions, and literally a make up full quarter of the entire value of the New York Stock Exchange, it’s going to be a bloodbath.

Remember how small the AI field was 6 years ago? It was purely the domain of academic research, fighting for scraps outside of a handful of companies big enough to invest in am AI engineer or two on the off chance they could make something useful for them. We’re probably looking at a correction back down to nearly that scale. People who have drank the coolaid will wake up one day and realize how shit the output of generative AI is compared to the average professional’s human work

Machine learning models have much different needs that crypto. Both run well on gaming GPUs and both run even better on much higher end GPUs, but ultimately machine learning models really really need fast memory because it loads the entire weights into graphics memory for processing. There’s some tools which will push it to system memory but these models are latency sensitive so crossing the CPU bus to pass 10s of gigabytes of data between the GPU and system memory is too much latency.

Machine learning also has the aspect of training vs inference, where the training portion will take a long time, will take less time with more/faster compute and you simply can’t do anything with the model while it’s training, meanwhile inference is still compute heavy it doesn’t require anywhere near as much as the training phase. So organizations will typically rent as much hardware as possible for the training phase to try to get the model running as quickly as possible so they can move on to making money as quickly as possible.

In terms of GPU availability this means they’re going to target high end GPUs, such as packing AI developer stations full of 4090s and whatever the heck Nvidia replaced the Tesla series with. Some of the new SOCs which have shared system/vram such as AMD’s and Apple’s new SOCs also fill a niche for AI developer and AI enthusiasts too since that enables large amounts of high speed video memory for relatively low cost. Realistically the biggest impact that AI is having on the Gaming GPU space is it’s changing the calculation that AMD, Nvidia and Intel are making when planning out their SKUs, so they’re likely being stingy on GPU memory specs for lower end GPUs to try to push anyone with specific AI models they’re looking to run to much more expensive GPUs

Huh I’d never looked into that! Thank you!

I mostly used zram because there’s an easy-peasy script in the Debian repos making it dead simple to setup and never looked deeper into it

Side thoughts in the middle of sentences are definitely weird in written form. Heck they get messy in spoken form too! Some punctuation to help the reader understand what’s being communicated can go a long way, and in the format of a forum discussion where folks will quickly tap out a brain fart from a 5" slab of plastic and glass, when I see what appear to be multiple sentences mashed together into one incoherent one, I’ll generally assume it’s a writing error, because folks don’t proof read, they aren’t writing literature with multiple drafts. They’re just quickly jotting down a thought or two and somethimes errors compound with that level of quick communication

Yikes you’re literally financing your hobby! Better financial move is to get a used system to start with (usually a used gaming PC can be had for like $500ish, and I’m sure there’s plenty of people online you can ask for help speccing something out), squirrel away money for a couple of years (I like to keep a dedicated savings account just for big purchases like tech upgrades. $40 biweekly dissearing into another account you don’t touch is $2k every 2 years, so a 4 year complete refresh cycle for 2 people) and buy when you feel like it. Good news is it’s a small enough amount of cash to easily right the financial ship but still yikes!

Are people throwing 128GB in for shits and giggles these days? I’m kinda out of the loop and still used to people arguing about if 32GB is excessive or not. Used to be only needed for media production and honestly it’s still debatable for pure gaming systems if more than 16GB is overkill

Its not just electricity but also water consumption, and noise pollution if not particulate pollution too.

They provide little benefit to the surrounding communities with very few jobs by design and just consume local resources at usually discounted bulk rates to sell a service that possibly nobody locally would be in the market for

Also the scariest part of this datacenter inflation is how much of these new data centers are going to be abandoned within the next 5 years when the AI bubble pops and suddenly the companies spending like crazy on datacenter growth need to cut back. There’ll be lots of big empty buildings outside of small towns costing taxpayers a ton of money, much like when any big box store closes up shop. You can either spend a ton of money tearing it down, a ton of money rebuilding it into something useful, a ton of money attracting another business which may or may not front the cost for remodeling the space or a ton of money maintaining the empty property so it doesn’t fall over and become even more of a blight. There’s no winning for these small municipalities that just get used and abused by large businesses

Zram on Linux is awesome! I’ve used it heavily in both memory constrained systems and systems with 16+GB of memory running very poorly optimized code

Running for example, Cities Skylines with 40GB of mods can easily lead to running memory usage being 20-30GB uncompressed. With zram I can load that same mod load out on a 16GB laptop with no swap and it won’t crash where it would crash for being out of memory before.

Another example is Proxmox with over-provisioned lxc containers. Since it’s still the kernel scheduler running all of the processes in those containers zram can keep them all running nicely even when a heavily modded Minecraft server gets a few players online and starts pushing past memory limits, where before I set it up I’d have some of the Minecraft server processes get killed to free up memory resources without warning or proper logging by Minecraft

Edit to add: my daughter’s first laptop has only 4GB of memory and runs a decade old Celeron booting from a spinning hard drive, the definition of budget ewaste. Zram makes it so it’s CPU limited running Minecraft rather than memory limited!

I mean since it’s driven by demand from companies running AI models it’s mostly impacting higher speed modules with lower speed/capacity kits being less significantly impacted. Granted that’s also pushing demand up for those lower speed kits since for gamers quantity matters far more than speed, meanwhile AI really cares about speed most

The good thing about new AM4 boards being available at this point in time is you have options to keep older hardware running. Usually the CPU and memory will out-survive motherboard. Much like those new Chinese motherboards supporting 4th and 6th gen Intel CPUs, this is great for longevity and reduces how much production is needed

In a sane world, the limitations of a CPU socket would be reached, and then newer SKUs would no longer be released

I’d argue that it would be best if computers were more like cars, a new platform gets released each decade or so, and small improvements are made to individual parts but the parts are largely interchangable within the platform and produced for a decade or two before production is retired. More interchangable parts, slower release cycle and more opportunities for repair instead of replacement