Posts by Nate Amsden

where are they going to get the money

Redoing all of that is going to cost some serious cash and seems like they were/are very short of funds. Guessing that so much old stuff was cheap to run as you don't have support and updates anymore. 3rd party hardware support on servers is probably 95% cheaper than full OEM hardware/software support. Legacy also likely means not many changes are happening as well perhaps out of fear of breaking stuff. Eventually it'll all fail of course (the legacy stuff). So those savings don't last forever.

Of course many on el reg have seen that in most cases cloud costs more. Certainly are situations where it makes sense. But it doesn't sound like they have a plan(or wasn't in the article) where the money will come from.(Or how much they expect to need/how long it will take). Seems like someone who doesn't know anything just says let's go cloud! Have dealt with several people like that over the years that don't know anything. Then are caught unprepared for the reality.

not too bad

I can think of only a couple of times in the past 15 years my ESXi systems needed a VM with a USB controller. And for workstation, I do use USB passthrough (on one VM) on that but not really concerned, if there is undetected malicious code there I have bigger things to worry about than VM escaping. I don't know why by default VMware assigns a USB controller to new windows systems in ESXi, I always remove it, never needed it.

Also on my linux systems I disable the framebuffer(https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-security/GUID-15D965F3-05E3-4E59-9F08-B305FDE672DD.html) to prevent any repeat exploits of that happening. On windows it's less useful without a framebuffer(assuming it's possible to have a functioning system at all with it disabled, not sure) so I leave it on of course but they make up a tiny fraction of the overall VMs in my environment.

That said, haven't had any known/detected malicious code on my systems since the [STONED] virus in the early 90s (excluding some seemingly harmless bad things. detected by virus scanners on game key generators and stuff in the late 90s).

I had to educate Oracle on their own software

Haven't had any serious Oracle software in a while but at one company went through two audits, back in 2006-2007. First audit I was a brand new employee so wasn't involved. Company ignored my advice to change from Oracle EE to Oracle SE telling me "we got it all figured out", until the 2nd audit where they got caught with their pants down again and had to pay up big (for them anyway). I recall being in that audit and educating the Oracle folks on Oracle SE CPU licensing vs per core on EE. They didn't believe me at first but later checked and verified I was right. I was quite surprised they were not aware of that basic thing at the time. The company originally licensed Oracle SE One (before I started), and their DB consultants deployed Oracle EE (Because that was their standard, perhaps they were never informed of the license requirements). After 2nd audit then I moved everything to Oracle SE and optimized things quite well, such as changing from Dual socket/Dual core processors to single socket/quad core (in a couple of cases HP had to replace the motherboards as they advertised their DL380G5 supported quad core, but the early generation did not, they later updated their docs to reflect this).

A couple jobs later I licensed a tiny Oracle SE 20 user something license for VMware vCenter(so I could run the DB on Linux, I had some history with Oracle unlike DB2 which was the other option, wasn't going to use MSSQL since I wanted Linux and that didn't run on Linux at the time). I believe I kept it in compliance without issue the whole time I used it(retired it when I deployed a new vCenter 6.5, and retired the 5.5). Though Oracle would often ask me what I was using Oracle for, when I told them they always shut up fast and never brought up any possibilities of audit.

Main thing I miss about Oracle DB anyway is enterprise manager's performance stuff, that was so cool. And at least with Oracle 10G you could use it (even though technically it required a EE license) with Oracle SE, so at that job in 2007 that is what I did, and I could wipe all evidence of it easily if needed with a single command. Oracle 11G or 11GR2 or whatever closed that licensing loophole, so at my next gig with the vCenter stuff I was unable to install that performance pack to enterprise manager. I haven't seen anything come within 1000 miles of being as useful as that instrumentation and the web based ease of use for identifying things since. Fortunately my day job is not DBA so I guess it's not my responsibility, still was so nice to be able to easily self serve information like that.

plenty of time

20 weeks to get your servers seems like plenty of time to prepare your datacenter ..

I remember the first company I was at that used HP servers. It was in 2003. Took a while to get stuff I guess I later learned mainly due to integration issues with compaq. So i remember we had every server shipped overnight.

reminds me of SGI cloudrack

15 years ago ...

https://www.theregister.com/2009/03/18/rackable_cloudrack_two/

Hardware wise anyway

"This time around, the trays don't include the power supply, which has been shifted out into the rack enclosure itself and which provides direct conversion from three-phase AC power coming out of the data center walls to 12V power needed by the servers on the tray. So, the "server" doesn't have a cover, doesn't have any fans, and doesn't have a power supply."

I was looking at using these at the time for my (then) org's first hadoop cluster. VP of group decided to go another direction cutting corners on quality and cost. I left shortly after. Was later told their new vendor had a 30% hardware failure rate and with quorum requirements it meant the first year of operation the cluster was at half capacity. I had a great laugh. Company is long dead now. I enjoyed the 2nd best commute though(short of working from home). The office was literally across the street from my apartment. I actually had co workers driving in parking further away to avoid paying parking fees.

found a way

They found a way...

"Hey procurement person - yeah you, no need to do any work this year do it next year."

Meanwhile I just checked I have a few HPE DL380Gen8s that just entered their 11th year of service in the past week. Serving important(but not critical) roles. And my oldest switches from when my org moved out of the cloud still in service(software support ended last year but hardware support continues to 2026) for 4,434 days according to their odometer readings (translates to 12.1 years), though I do plan to replace both sets of these old systems this year. The switches are in critical roles, though are in a redundant configuration (Extreme Networks X670V-48x).

I'm sure others have older of course, but 5 years is a very low bar.

factory reset only if you are hacked

Unless you are super paranoid I guess. Their more in depth docs are more clear

https://forums.ivanti.com/s/article/Recovery-Steps-Related-to-CVE-2023-46805-and-CVE-2024-21887?language=en_US

"If your ICT scans were clean, you are not affected by this activity."

Basically if the external integrity check fails then you should factory reset.

Their new mitigation to work around the SAML thing seems to break SAML entirely(fortunately I don't use that..yet. But Duo is forcing everyone to SAML soon who uses Ivanti Secure, at least those that want the fancy Duo UI integration).

Wondering on the backlash

I wasn't quite up to speed on this as I assumed I was but got myself up to speed now(via https://nandresearch.com/research-brief-impact-of-new-vmware-bundles-pricing/). After learning what I learned regarding the bundles and no longer selling the standalone products(which I was surprised to see after checking both Dell and HPE's site for vmware related things and not finding anything), I have to think back to VMware's "vRAM tax" stuff they tried a decade or so ago, which they backtracked on eventually(took a while for them to realize this, and fortunately never affected me as I was on an older version and didn't upgrade until a while after things reversed).

Wondering after a year or so if a bunch of customers jump ship would Broadcom reverse course(maybe not entirely, but to some degree), maybe or maybe not I don't know all depends on the market reaction.

does juniper do much in the "AI" space?

I have been watching the networking space off and on since about 2003. Juniper tried to buy Extreme way back around that time before deciding to go it alone and make their own switches. I remember their first 10G 1U switch, didn't even run their software they OEM'd some 3rd party platform and put their name on it. They came out with their own stuff of course eventually.

As time went on it seemed they struggled to gain much share outside of their service provider space(basically they were mostly selling to people already buying their routers). Haven't noticed much has changed in that regard, so seems strange to me that HPE would place this kind of a bet with "AI" as a justification. Maybe it's just due to the hype surrounding it, makes it more likely their shareholders approve of the purchase or something.

Juniper certainly has a lot of solid tech though the complexity of their JunOS is pretty crazy to me(same goes for Cisco). I'm sure it makes sense for several use cases though.

As someone who does networking as only a minor part of my role(though through every company I have worked for in the last 20 years my networking expertise was above most everyone else's in the org), Extreme's simplicity and functionality beat everything else on the market(been using them since 1999). Hell I still use network designs I first started with in 2004 because they work so well(such designs wouldn't apply to "AI" workloads but I don't have any of those). Not that Extreme has ever really taken off in the market either, I remember a Foundry Networks rep back in 2004 trying to convince me Extreme was going out of business in the next couple of years.. ironic that it was Extreme who acquired several of the Brocade(Foundry) network assets over a decade later.

HPE seems to have so many different switching platforms under their roof, hopefully they can consolidate the user interfaces at least(pretty radical differences between some of them anyway).

Certainly seems to me if AI networking was a super important thing to HPE they could do a much better job sourcing the same merchant silicon ASICs and making their own high end stuff for a fraction of the money. They obviously have the server/storage(and even networking, Aruba switching is super popular among network engineers(I have never personally tried it)), market penetration already.

Curious for anyone who knows - does HPE have (m)any supercomputers (Cray, etc?) out there running on Juniper network gear? I assume a lot of it is infiniband or something. Curious because I'd expect similar networking requirements for AI.

Looking at Juniper's stock price since the dot com era, I was quite surprised how little it has appeared to move in the past 20 years.

weird

Seems all of this should be directed at the companies(Banks or whomever) that are using the providers. The providers have SLAs, and it's up to the customer to determine what SLA is acceptable and how to plan for (or IF to plan for) breaches of that SLA. Even more important for most cloud providers as I have been saying for 12 years now they are "built to fail", meaning you have to build with failure in mind, much more so than on prem (which more often has more redundancy built in whether it is redundant storage, vmware high availability, fault tolerance, high availability networking etc). Things fail in cloud 10x more often than on prem in my 20 years of experience.

You direct the businesses of Singapore that fall under this to have to maintain some kind of SLA then it's on them to choose providers/design/etc in order to try to meet that SLA.

If you try to enforce better SLA terms on the major cloud companies they will likely just laugh at you. Even Microsoft tries to tell their Office365 customers to keep backups of the Office365 data as MS is not responsible for that.

The worst uptime of any company I've worked for was about 20 years ago for me, company I was at provided online transaction processing for major U.S. telco mobile transactions. Our SLA was 99.5% of unscheduled downtime per month, that EXCLUDED 12 hours of scheduled downtime for software upgrades each month. We missed the 99.5% SLA most of the time anyway I think, mostly due to bad software design. I recall one ~30 hour storage array outage, another ~20-30 hour Oracle DB outage(Oracle flew on site, and said there is nobody else in the world doing what we were doing, we were doing it wrong, and to fix our app), another ~12 hour internet outage (due to bad BGP routes sending our traffic to Russia resulting in 99% packet loss), many multi hour BEA Weblogic outages and dozens of others. Learned a lot(critical to my future career, didn't realize it at the time), had fun, burned out hard core though took years to recover. Despite all of those outages never had a single blip in data center availability. If you bought any games/apps/ringtones/wallpapers/etc on the major U.S. carriers 20 years ago those transactions flowed through our systems.

dselect

Debian had dselect before apt, and it handled dependencies, it was a lot more painful to work with though for sure. I still remember on one install spending about 6 hours going through the various packages in dselect to find what I wanted because after the install was done I didn't know how to start that tool again(or even what the name of it was). Though back in those days I still built a lot of things from upstream source directly(including KDE, GCC, Gnome, X11, kernels(last kernels I regularly built were 2.2.x)).

For me, apt started to become much more useful with Debian 3 which for whatever reason I believe I recall as having a huge jump in number of packages available(the first huge jump in my memory at least), it was the only Debian version where I used "testing" for several months (instead of "stable"). Reading the release notes of Debian 3(tried to find the number of packages), they specifically said apt-get for upgrading the (distribution) version was not recommended(and to use dselect instead).

Even though dselect has been depreciated for over 20 years it still serves a critical role for me for copying package states between systems(not something I do often but it is handy)

dpkg --get-selections >selections.txt

(copy selections.txt to another system

dpkg --set-selections <selections.txt

apt-get dselect-upgrade

Perhaps there is another way to do it these days. I recall at one point trying to find a similar process on a RPM-based system(don't recall which), but was not able to.

My systems at work are pretty much all built with configuration management software so little or no need to do that process there, but my ~20ish home linux systems(including my public email/web/dns) are managed ad hoc still.

missing competitor

Looks like Opencart is an open source shopping thing. So I'd say their biggest competitor is Magento (which has open source, commercial, and hosted offerings I think), not those other SaaS platforms mentioned(though they could be the biggest in the space). Part of the point of having the source is being able to customize it beyond what SaaS allows.

I worked at a couple of orgs that used Magento enterprise(hosted on our own gear), though the last version that we used was 1.14 I think (which is probably 8-10 years old now).

saw an interesting comment from a person on linkedin a month ago, unsure if the info came directly from them or if they quoted another source

"Since its inception, Splunk has incurred net losses yearly, resulting in an accumulated deficit of $4.05 billion. Splunk has bet its future on costly cloud services that require continual infrastructure investments. Splunk’s $3.099 billion in debt exceeds its annual revenue. "

Which surprised me, I assumed Splunk was doing much better than continuing to make losses every year since they started, hard to imagine what they could do to get that debt under control in that situation(other than get bought I suppose).

wonder if they will have a repo at some point

MS edge for linux installs a config so apt can download updates direct from https://packages.microsoft.com/repos/edge/ . Firefox is my main browser though I did switch to edge for MS teams on linux as well as OWA(prefer to have things in different browsers, in the past I used another instance of firefox, as well as Seamonkey).

After switching back to Firefox from Palemoon(after palemoon decided to break all of the addons though I think they reverted that decision later) have been with ESR ever since(took some time getting over losing my addons that I had used for a decade or more many of which would never be updated again), manually install to /usr/local/firefox-<version> and symlink /usr/local/firefox to that. I guess no proper OS integration, though I run my firefox as a different user account "sudo -u firefox -P -H VDPAU_NVIDIA_NO_OVERLAY=1 /usr/local/firefox/firefox %u" (unsure if that NVIDIA option is still needed these days or not(though still get video tearing, eventually realized it wasn't a linux issue as there is tearing in video on windows as well on the same hardware).

Edge runs under another account as well. Was really annoying to configure Pulse Audio to work right, but eventually figured it out.

wondering if these exploits work...

..against Citrix VPN systems that have no users connected? Seems like exploitation requires at least one user session to be active. I disabled my main access gateway("disable vpn vserver <name>") a few months ago(following the last exploit), I did used to use access gateway, and plan to get back to using it again, but upgrading from Netscaler 12.1 -> 13.0 broke duo authentication and I haven't gotten round to fixing it yet, so was sitting without anyone connected to it for a few months (I was the only one that used it, everyone else uses another VPN which I am now using as well, though liked Citrix more)

Netscalers by default I believe come with a 5 user license, so wouldn't surprise me if there are a lot of systems out there that may have VPN configured just don't get much usage, in my case I kept it alive over the years primarily as a backup VPN for other IT staff if there was an issue with the primary system (which uses a completely different vendor/product), didn't cost anything since we needed the load balancers for load balancing anyway. I still patched anyway, even though likely the exploits would be completely ineffective as my only vpn vservers were disabled.

forced to explain to customers...

"Yeah, we realized a bit too late that cloud is really expensive and not worth it since we don't have unlimited money, and are unable to charge a fee that would allow us to operate that way and still have any customers".

Nice to see more people realize this, though perhaps too late for MariaDB time will tell. I've used MariaDB for several years after migrating from Percona after Percona jacked up their fees by about 10x many years ago out of nowhere. Though not in cloud, and not paying for any support. Seems to be a fine product but I'm not a DBA, though sometimes play one on TV.

only a matter of time

before google removes the ability to disable it. Of course if you really cared about privacy then you wouldn't be using Chrome to begin with. Sadly most people don't care(even techies).

what are those ports?

looked them up myself

"The Alveo UL3524 card hosts four 8-lane small form-factor pluggable (QSFP-DD) connectors that are housed within a ganged 1x4 QSFP-DD cage assembly with heatsink. It can be populated with QSFP or QSFP-DD direct attach copper or optical modules supporting up to 7W. The QSFP-DD can connect interfaces up to 100G using optical modules or cables. A 161.1328125 MHz clock is provided to the QSFP-DD interface such that different Ethernet IP cores can be enabled."

https://docs.xilinx.com/r/en-US/ds1009-ul3524/Network-Interfaces

reminds me of

Executor, still seems I have my copy of 2.0W from 1998. Seems it went open source at one point and is forked and still being developed

https://www.emaculation.com/doku.php/executor (history)

https://github.com/autc04/executor (current)

"Executor was a commercially available Mac emulator in the 90s - what makes it different from other Mac emulators is that it doesn't need a ROM file or any original Apple software to run; rather, it attempts to re-implement the classic Mac OS APIs, just as WINE does for Windows." (from the 2nd website)

Last time I tried bochs (directly anyway, unsure if anything I've used since has used bochs as backend) was 1998 too, it was cool to see but performance made it unusable for me at the time, fortunately VMware for linux came out a short time later..

been a long time

I was only ever really exposed to Bugzilla as a user once at one of my first jobs back around 2001. At least at that point it seemed like an unusable mess, no positive memories of it. First started using Jira in 2006(I think) and it was really nice for the basic things we needed it for (I'm in tech operations not software dev), though the developers used it too. Have been using Jira (and confluence) ever since across every job(the solutions were always in place before I was hired).

Though it is true that IMO the Jira/Confluence UI has gone crazy over the past 5-7 years(maybe more, I lost track), for me mostly negative feedback. At least they seemed to have stopped their original plans for trashing the "legacy editor" on confluence. For a couple of years they were really marching towards that saying everything had to be migrated, even when the new editor didn't work well(and still has some glaring issues today that remain unfixed especially with table auto sizing which worked fine by contrast on the old editor). My favorite version of confluence was I think v3 ? The last one where you could still edit the wiki markup.

At my last gig at one point the management tried pushing for this "kanban" board BS, which was just a waste of time and fortunately died on it's own pretty quickly.

I use Xwiki at home which is pretty good.