Recent searches
Search options
Elon Musk's favorite supposed data expert, who he's retweeted at least a dozen times, claims she can only process 60,000 rows of data before her "hard drive overheats".
Perhaps someone should rescue her from where she's apparently stuck twenty years in the past, which is the only possible explanation for those hardware limitations and the apparent lack of access to cloud compute.
Unless, of course, she's just completely making shit up.
Why Not Zoidberg? 
@molly0xfff Bet she uses extra powerful secret AI code rows...
@molly0xfff You've gotta be doing something specially dumb to have 60k rows trip you up. I've played with spreadsheets with more rows....
@molly0xfff (I was doing something particularly dumb)
@LovesTha @molly0xfff whoa whoa whoa, how do your spreadsheets go beyond 65536 rows? /s
(The quoted number is suspiciously close to this)
@GuillaumeRossolini @molly0xfff custom compiled version of one of the floss spread sheet apps. I think I got close to 1M rows before I gave up and did it a proper way.
@GuillaumeRossolini @molly0xfff (yes I saw the /s)
@GuillaumeRossolini @LovesTha @molly0xfff I've used a spreadsheet where rows from 65536 onwards were wrong, but that was a bug in the tool that generated the spreadsheet from another data source.
Excel from "twenty years in the past" might not handle it though.
@armb @LovesTha @molly0xfff yeah that’s a given, but if this person is using excel (and so outdated), that’s not a great look 
[edit] as it relates to their claim of hard drive overheating, even if in jest
@GuillaumeRossolini @LovesTha @molly0xfff dude, have you even heard about google spreadsheets?
@LovesTha @molly0xfff the old 32-bit Excel limit was 64k, right? Maybe they are stuck on Office 2005. Which is funny in so many ways in my Head. Starting from 'Excel?' to my new head canon that they rejected to upgrade because they hated the ribbon layout.
@develin @molly0xfff have they removed that limitation yet? I know libreOffice copied that limit so people abusing the roll over had compatibility.
@LovesTha Yeah, 64-bit always had a higher limit, so current Excel supports up to 1 million rows. Not that it is fun to use that large a file for anything meaningful. A few VLOOKUPs on such a large file kill your PC :D
@LovesTha @develin @molly0xfff Both Excel and LibreOffice Calc now support 1048576 rows.
@develin @LovesTha @molly0xfff
right but this person probably wasn't born in 2005.
@molly0xfff The *hard-drive* is what overheated?
@aspensmonster @molly0xfff yeah, right after she downloaded more RAMz
@aspensmonster @molly0xfff Claerly Overclocked the HDD from 2700rpm to 5400. And sadly no cooler mounts in the front.
Spinning Rust FTW!!!!1
@aspensmonster @molly0xfff I'm guessing she's calling the tower "hard drive" – a lot of clueless users do this.
@molly0xfff Is DOGE too "efficient" to pay for decent workstations? Wouldn't surprise me, given that management consulting has already infested their roster.
@molly0xfff processing data in hell require so much extra cooling. 
@buherator @molly0xfff I am suspecting that you have an "excel" data expert...
Feels like AI Chatbots making up stories.
@buherator @molly0xfff Got that blue checkmark, of course.
@buherator @molly0xfff Hmmm
- large data set
- 60k rows
Pick one, it can't be both.
@buherator @molly0xfff 60k rows is a telltale Excel mark, so either they try looking credible for beginners or it’s indeed a “fact” easily parroted by AI
Wait, what?
How poorly set up is her database that she can only run 60k rows? I can do that trivially on sqlite running on a raspberry pi.
My assumption is that she's simply lying, but I'm now awake pre-6am wondering if it would be possible to do that on purpose with modern hardware and software. Even indexing improperly and then doing joins on string columns wouldn't give that sort of anti-performance.
@passenger Perhaps if she moved the hard drive from where she left it on the radiator...
Even Excel can do better than that
Wait
Wait
Fuck
You're right!
Excel 97-2003 .xls files have a maximum size of 65,536 rows and 256 columns.
I really hope this is not relevant information.
@passenger @JessTheUnstill @molly0xfff Chances are they grabbed the data export, but saved it as a .XLS file, so there are only ~65k entries.
Edit: Older corporate DB export interfaces usually only exports in CSV, XLS, (and bizarrely, pdf.)
I remember coming up against these sorts of restrictions with older versions of Microstrategy. Briefly. Before we bullied IT into giving us proper query access.
If she's pulling data via an export dialogue on a mouse-driven GUI, then that implies that Musk and his boys haven't given her the access to query the database directly like a professional. That's hilarious and I am going to be laughing about that all day.
@passenger I used to have to pull like 5 of these exports from all sorts of different systems at [famous tech megacorp] to try to cobble together what we (or the larger corp) actually sold people. Direct query access was entirely out of the question. I made the .xls export mistake before.
I learned to write Python scripts just to merge and clean the data so I didn't have to do it manually every month. A year later, they reordered and renamed the columns in the primary default export. I cried.
Ooof! That reordering and renaming must have hurt.
One of my first jobs when I got into data was combining several such sheets to produce a dataset that we then use downstream. Unfortunately one of those sheets was produced at least partially by hand, I think, because that's the only explanation for why column headers occasionally unpredictably had merged cells some weeks and didn't have them in other weeks. That taught me a lot about error checking. It was also very painful.
@mayintoronto @passenger That reminded me of a late 90s gig where we built a system to track and automate some manufacturing processes. The owner was an ex mainframe developer who would occasionally go into the DB and "fix" the names of some of the columns....
Half the code in the system ended up being validation of the structure and automatic correction for these changes...
@mayintoronto @passenger I, a non technical person who is baffled by any kind of even remotely advanced tech talk, happily goofed around with 100,000 row datasets as part of my last job. On a laptop. That did not overheat.
@AlexanderVI @passenger Congratulations, you're technical now. The big boss at the aforementioned company would sometimes ask me for tech support because I could use Excel.
Yes, this was a software company. 
@mayintoronto @passenger Build yourself a data warehouse and you can successfully merge all of those data sources.
@alterelefant You're very funny. That's something that could never happen at the megacorp level.
@mayintoronto @passenger Instead they rather pay 150 k for some external tooling that does exactly the same thing for ten times the price.
@mayintoronto @passenger @JessTheUnstill @molly0xfff I suspect that the pdf export is a feature.
Union : according to the labor agreement you have to give us the annual promotion/salary/statistic/... update so we can check you did not screw things up.
Evil company: oh yeah, sure, here is a 99765 pages, 8pt font, mangled pdf database export !
My money is on audit and compliance.
I have had numerous fights with compliance that wants all the evidence in screenshots or .PDF ... Because apparently they consider that harder to fake? Idk, they're weird.
@b3nb3n @mayintoronto @passenger @JessTheUnstill @molly0xfff That’s the exact format of the pay scale appendix on my most recent contract. Scaled down to print in a roughly A5 size booklet.
Fortunately, OCR is pretty decent these days
@eliterrell @b3nb3n @mayintoronto @passenger @molly0xfff
@b3nb3n @mayintoronto @passenger @JessTheUnstill @molly0xfff That is extremely evil indeed.
It would make me laugh so hard if it wasn't that her hard drive was churning, she just hit the bottom of Excel, and so the data stopped loading.
@passenger @molly0xfff it’s such a pathetic lying number, too
most multi-year group chats on thousands of people’s phones would have that many messages in their databases
many years-old email addresses (or people on many MLs) have deep into the 6 figures worth of email items in accounts
60k is such a piddly small number you can have an esp32 run a db server for it (with light queries)
my bet is “number picked to sound big to people who know nothing”
@passenger @molly0xfff "Using unsigned short for row IDs on data that doesn't even need row IDs to own the libs."
@passenger @molly0xfff seriously, you could rip through 60,000 line flat files on virtually any desktop built in the last 20 years without really breaking a sweat, unless the lines are a megabyte each, it which case “data expert” is not what I would think of this person.
@lerxst @passenger @molly0xfff My son's thesis processing read the metadata of 14m works into Pandas/Python/Jupiter on a MacBook Air M1 in under two minutes.
64K of records suggests an ancient spreadsheet rather than a modern workflow. A hardrive overheating after what would be around 256MB is laughable. God knows what that computer does when faced with a video file.
@glent @passenger @molly0xfff it’s scary how fast the new Apple architecture is, especially on comparative power usage. I treated myself to a new M2 MBP a couple years ago and I didn’t expect it to be so good that it could replace my circa 2018 desktop PC at the heart of my studio. But it was!
@lerxst @passenger @molly0xfff The only issue we had for student 'big data' use was that the supplied power supply wasn't big enough to both run the computer continuously at full rate and charge the battery from empty. So we had to buy a larger power supply. I notice that Apple in Australia now offer purchasers a choice of smallest possible or biggest usable power supply.
@passenger @molly0xfff I mean it's technically possible to thrash your hard drive for a very long time for only 60,000 rows if you write a select query dumb enough to table scan your big table multiple times with really bad joins...
But you'd have to work hard at it - or more likely use ai to write the really dumb sql query for you.
But none of that would "overheat" a hard drive...
She's using the new Tesla-brand hard drive, which shares the unique engineering characteristic with the rockets and cars, of spontaneously bursting into flames.
@molly0xfff How can she do a full run now surely her computer will melt?