Posts by thames

Re: Wasn't this already settled?

The "tainted" bit was just to tell the kernel developers that there was a proprietary close source module loaded into the kernel, so that the kernel developers wouldn't waste time working on bug reports for problems they had no source code for. I think the main reason for this originally had to do with the numerous bugs in NVidia's proprietary video driver (which was the number one cause of Windows crashes too), but they made the solution generic to cover other cases as well.

The basic idea is that if you submit a bug report and the "tainted" bit is set, the kernel developers will tell you they aren't going to waste their time on it. If you can reproduce the problem without using proprietary closed source drivers, then they'll accept the report.

The above doesn't affect ordinary users since they are normally getting their software from their distro, not direct from the kernel developers. Your distro is the one to file bug reports with. The kernel developers on the other hand are dealing with the distros and with various hardware vendors, and the latter in particular are often trying to push bugs they created off onto someone else to solve for them.

Re: Whare are the NSA / GCHQ whe you need them?

From what is said in the story, it sounds like the PCs are used to schedule tests and send out results. Air gapping them would have the same effect all the time as the problem they are experiencing now temporarily. Talk about an own goal.

Re: Futile gesture

@Philip Clarke - "Do you remember when encryption could not be exported?:

Yes I do. I remember that the US became a backwater for encryption development as companies in the rest of the world were freed from competition with American companies. Today, the overwhelming majority of encryption products come from companies outside of the US. If you have followed the IT news in the past few days, you would have seen articles on this.

Re: Back to the Nineties

The current company is actually "SCOG" (SCO Group), which is not the same company as the original SCO. The history went something like this:

* SCO and IBM entered into a joint venture to develop and sell a version of proprietary Unix. This version was supposed to cover the market from the small x86 box sector up to large RISC servers. IBM would contribute the large system expertise, and SCO would contribute the x86 box expertise (including marketing channels). Both parties hoped this would establish their joint brand of Unix as the industry standard.

* Meanwhile, Linux was starting to make inroads into the unix market, particularly at the lower end. Companies involved in selling and supporting it included Red Hat, Suse, and Caldera.

* SCO eventually saw the writing on the wall with regards to Linux, and decided to give up on the joint venture with IBM. They sold their unix business off to Caldera, who planned on converting the SCO Unix customers into Caldera Linux customers.

* When SCO pulled out, IBM exercised an option in the contract which let either party terminate the joint venture if the other party sold its interest. SCOG's current lawsuits against IBM revolve around this event.

* SCO sold themselves to Sun, who wanted them for their remaining software products. Sun of course were later bought by Oracle.

* The dot-bomb hit, tech stocks collapsed, and Caldera couldn't get free money from the stock market any more. Meanwhile Caldera was also not having a lot of luck selling their brand of Linux to their new customer base. Customers were opting to switch to Red Hat instead. They decided they needed a new business strategy. At some point in this Caldera themselves came under new ownership in circumstances which are as hazy and odd as their subsequent behaviour.

* Caldera renamed itself the "SCO Group" (SCOG), brought in new management, and went on a sue-world-plus-dog campaign using financing from certain companies who had an interest in seeing Linux dead.

* SCOG represented themselves as "owning" Unix. Novell, who in fact did (and still do) own Unix, having bought it from AT&T, disabused them of that notion in court. SCOG was just another Unix licensee like IBM, Sun, HP, etc. They also had a contract with Novell to act at the outsourced agent for collecting license royalties from the other licensees, and it was this role which they tried to use to represent themselves has having legal standing to sue others. Novell took them to court over this issue.

SCOG's lawsuits with IBM revolved around several issues.

* One was the termination of the above mentioned joint venture. This is still active.

* Another was over IBM's JFS file system, which although written and owned by IBM, SCOG claimed that IBM could not put this into Linux because of "wave hands and shout loudly" reasons. Unfortunately for SCOG, there were two different versions of JSF. The one which went into Linux actually came from OS/2, not AIX. This was the closest which any of SCOG's cases came to actually involving Linux, and it was something which would have had little impact outside of IBM even if SCOG had somehow won.

* IBM counter-sued SCOG over various issues that I can't remember, other than that they were narrow technical matters that would have been open and shut cases that would have been difficult to dodge if they came to trial.

As a footnote, when SCOG declared bankruptcy, they had to list all their creditors. Imagine the amusement we had when one of these creditors was a well known "independent industry analyst" who had been in the lead of trumpeting how solid SCOG's case was, and how Linux was doomed.

Re: It's a question of political will

@Anonymous Coward - "I've struggled for years trying to get above 2Mbit in Canada,"

I don't know of any major ISP in my area in Canada that's even selling anything that slow these days. The slowest connection from someone like Bell you can buy these days is 15 Mbps. Meanwhile they're calling me all the time and knocking on my door trying to flog me 1 Gbps fibre. I'm in a smaller city by the way.

If you had 2 Mbps in any recent years in Canada, then either you were off on the end of a semi-rural line or you had other hardware problems, possibly in your on-premises wiring or local loop. A high error rate will give you what is effectively a slow speed. That's the problem with anecdotes, they tend to be from the one person who had a problem and was unhappy about it, rather than what the overwhelming majority experiences.

I'm with a smaller ISP who have an extra cheap 5 Mbps package (along with faster ones). I've picked that one because I don't download movies, and even 5 Mbps is more than fast enough for Youtube, web browsing, Github, email, and all the other stuff I do. And I'm tight with my money as you may guess.

As for 2 Mbps? Never heard of it from anyone.

Re: Interesting

The mobile/tablet version will be a Unity GUI with a Mir window manager. This is the reason for Unity - it's intended as a GUI which has features which can adapt to both desktop and touch. That doesn't mean it's exactly the same GUI - but rather there are common design features which are used to create both the desktop and small screen touch versions so that "apps" designed for one will work in both and users familiar with one will have an easy time adapting to the other.

Apple uses totally different GUIs on desktop and phone/tablet. Microsoft tried sticking a phone GUI on a desktop (Windows 8). Ubuntu is taking a third path - a GUI which is a desktop GUI when working as a desktop and a phone/tablet GUI when working as a phone or tablet and automatically switches as required. I suspect that both Apple and Microsoft will follow in this path eventually, and indeed I believe that Microsoft have already decided they will need to do so.

This by the way is the main reason why there was the split in Linux desktops between Unity, KDE, and Gnome. Ubuntu wanted to focus on touch (tablet/phone) first with desktop being an offshoot of that. KDE wanted separate touch and desktop GUIs. Gnome (basically Red Hat) wasn't interested in tablet or mobile, and so didn't want to accommodate it except as a afterthought. Mint with Cinnamon and MATE are also in the "who cares about anything but desktop" camp.

Canonical think that free/open source operating systems of the kind we are familiar with have only a narrow window of opportunity to establish themselves if they are not to be squeezed out by changes in the market. Replacing one monopolist (Microsoft) with another (Google) is no improvement, and the new generation of phone and tablet hardware is increasingly locked down so you can't install any alternatives.

It's probably too late for anyone to unseat Android from market dominance in the phone and tablet field. It's probably as unshiftable there as as Microsoft is from business desktops. This is why Ubuntu is putting the priority on "converged devices", which is what they had been banging on about for years before Microsoft woke up and noticed it.

I suspect that even for dedicated desktops, the conventional "Wintel" PC hardware we see today will largely disappear from the market. It will be replaced by very cheap and very small commodity boxes about the size of a Raspberry Pi (with a plastic case of course), and using a similar physical layout (all on one board). Everything including CPU, RAM, and SSD will be soldered onto a very small PC board. Whether it will have an ARM or Intel CPU is an open question. I wouldn't want to bet any money on Microsoft still being a significant factor in the desktop OS market by then either.

I suspect that the days of being able to go into a shop and buy what we see as a "conventional" desktop are numbered. If GNU/Linux (as opposed to Android/Linux) operating systems are not well established in the changed market by then it will likely be very difficult to buy any non-server hardware which will run it.

With Ubuntu (14.04), when I built a new PC I just stuffed the DVD into the drive, and it booted up and installed just like it always did. I didn't touch the firmware. There was no drama or fiddling with anything.

I'm not a fan of UEFI however, for reasons that have to do with security. Matthew Garret wrote a blog post on it a few years ago when he was working on UEFI support for Linux, and he mentioned that there were more lines of code in UEFI than there were in the Linux kernel, if you exclude drivers (to get an apples to apples comparison). On the other hand, Intel had no plans for security support, and the motherboard makers had no way of coordinating and pushing out security fixes. Garret found a security hole, and when he contacted Intel to report it and ask what their plans for fixing it were, their reply was "none".

Personally, I would prefer something like Coreboot for most applications. It's small and simple, and so presents less of a security maintenance problem. It's also what a number of popular Chromebooks ship as their firmware, so it obviously works. Put the complicated stuff in the OS, as they already have established mechanisms to handle security fixes for the bugs which are inevitable in any complex system.

Re: You don't hang these things off the Internet?

@Walter Bishop - Yes, but you better be doing it though a fairly decent VPN, because there is no authentication or encryption built into the Modbus/TCP protocol. It will accept and execute any commands sent to it from anyone.

Modbus commands fall into four classes - read the state of one or more bits, set the state of one or more bits, read one or more 16 bit memory address, and set one or more 16 bit memory address. The Modbus protocol like most other industrial protocols is intended to run on very simple, very low end hardware. Modbus (serial) and Modbus/TCP are very popular protocols in industry due to their simplicity, widespread industry support, and the fact that it is an open protocol.

Think of Modbus devices as being like a PLC's equivalent to keyboard, mouse speakers, and monitor. Your keyboard does not authenticate with your PC or encrypt its communications. It expects to be plugged straight into the PC. If you route your keyboard to PC connection through the Internet, you better not expect the keyboard to be providing the security for this. The PLC equivalent is switches, valves, relays, and operator panels.

It's probably just as well that most industrial control system vendors don't try to built in much if any control protocol security to all their products, as getting it right is very difficult and best left up to the experts. As a result, if you want to read a Modbus device from the other side of the world then you need to use a separate third party security system somewhere upstream, as I said in my original post.

The big advantage that Modbus/TCP has over many of the protocols based on more proprietary systems is that it is designed to work with standards and technology from the IT industry which means you can use security hardware, expertise, and services from IT specialists who deal with security as their bread and butter.

Re: Nothing Special

The problem is basically the design and configuration of the bug tracking system itself. "Filtering" is a very generic term that I used to cover a lot of UI and process ground.

The bug tracker should be guiding the users to enter useful information while also providing a way for people to enter "me too" responses in a way which is obviously just a "me too". A "me too" response is useful in gauging how many users are affected in order to prioritise fixes, but not if people have to read and summarise everything themselves.

The actual letter is linked in the story, and there are three short bullet points listing what they want addressed.

Issues are often missing crucial information such as what the version is or how to reproduce the problem. The developers want Github to let them add custom fields and mandatory templates to ensure the user fills out all the fields.

For just simple "me too" responses they want a voting system so that users can up vote an existing issue rather than adding another comment with no real information.

They also want individual project contribution guidelines to be made more obvious to people making pull requests.

I suspect that the first two are the most important, since in my own experience getting useful information out of users can be like pulling teeth if they are not familiar with how to make a bug report that can be turned into actionable items.

Re: Are "electronic" components involved in the failure?

I don't even know what the cable looks like since I've never seen an MS Surface or know anyone who has one. As long as we are speculating wildly though, the most likely point of failure is where the wire comes out of the terminals at either end. In that case the problem could be insufficient strain relief, poor design of the connection in general, or a combination of that with the type of crimp connection or too much solder wicking if the ends of the leads were solder dipped or the wire is simply too hard or as too few (too thick) strands.

If you coil the lead tightly, the wire could be breaking at the terminals or pulling loose due to any of the above. If you want to look at things in a more general sense, it is likely that the root cause is someone's cost saving idea ran up against the hard reality of life that wire leads face in the field.

Wire leads are a commodity item, but there are different grades at different prices and the device product designer has to know what choices to make when specifying one over another. The lead manufacturer will simply give you whatever you ask for because well, they're not in the device business and aren't really expected to know what the end user really needs. That's why they get paid peanuts for their effort.

P.S. - In a past life I was involved in the design of manufacturing equipment for various types of leads. There's loads of different ways of designing and manufacturing them using different material, all with different pros and cons depending upon what they will be used for and what the customer (product OEM) is willing to pay for.

Right, because 32 bits ought to be enough for anyone.

@Asterix the Gaul - "Comparing AMD\INTEL over the last 2 decades shows that AMD have always trailed INTEL, not just on the architecture,"

Right, because 32 bits ought to be enough for anyone. And if you really need 64 bits, well there's always the Intel Itanic, which dominates industry market share. I heard that AMD was planning to respond to Itanic by coming up with some sort of 64 bit extensions to the x86 architecture. I wonder what ever happened to that?

Re: The money

Scopes are really just Ubuntu's take on copying Apple's Spotlight. What Ubuntu added was the ability for you to install third party plug-ins in a sandbox rather than just hard-wiring in a few sources. I doubt that the "scopes" idea had anything to do with money for Canonical. Most of the scopes they added were not money generators in any way and I doubt the Amazon one had potential for non-trivial (for a company the size of Canonical) amounts of money.

As a further anchor for the sort of numbers we are talking about, I recall reading that the search revenue that Linux Mint was getting from their own search customisation mods (putting a custom advertising header in Firefox) was generating derisory amounts of money. Of course Mint has a much smaller market share than Ubuntu, but in either case we are talking about very little money, unless you happen to be a one man operation.

The idea of including Canonical written scopes was to act as examples in order to get third parties to write them, so Canonical did a few which showed that there was potential for revenue. The idea was that when you searched for something, it would also do things like search Github if you had a Github scope installed.

I think that scopes were actually mainly meant for their phone platform, which is why they added things like an Amazon scope, which is a consumer oriented feature. They were using the desktop distro to try to kick-start the third party scope market for the phone. Scopes were supposed to be able to be extended to do all sorts of weird and useful things beyond just searching for stuff, although I can't remember the details (I didn't care enough to look into it further).

Pretty much anything new that Canonical adds to Ubuntu can be found in MacOS or OS/X, with enough of a change to avoid getting sued by Apple. The third party plug-in aspect of Scopes might be different, but the idea of sending search information off to to the Internet isn't. Apple fires your search info off to Microsoft via deal for Bing search.

By the way, to turn off Scopes, you can either un-install whatever specific Scopes you don't want via Software Centre, or just turn them off globally by going to Settings ==> Security & Privacy ==> Search (tab) and then turn the switch to off (when searching in the Dash, include on line search results). That's for 14.04, which is the current LTS release.

You can also turn off the feature which tracks your most recently used files and applications (which is not part of Scopes), exclude specific things or file types (I haven't used this), clear data, or turn it off altogether. The same applies to diagnostic reports.

I turned the feature off a couple of years ago as I have zero interest in it. I can say though that if we ever want an Internet where all the traffic isn't funnelled through a handful of big companies like Google, Facebook, and Microsoft (if they don't bin Bing as a hopeless cause that is), we'll need something like Scopes to give us a distributed search ability.

Re: Trust?

@tom dial - "Stipulating that equipment shipped from the US might be subject to interception and modification, the same certainly is true of similar equipment shipped from non-US addresses. (...) On the other hand, interception and modification by a government agency in the receiving country also would be a possibility, one not under the control of either sender or receiver, and that is not one about which either has a choice."

My own government could simply arrest me and grill me in the police station in order to find out whatever they want to know. They''re also unlikely to engage in economic espionage which will hurt their own economy. Foreign countries such as the US on the other hand can't simply send the police around, nor do they have any problems with engaging in economic espionage (e.g. against Airbus as was discovered in an EU investigation as long ago as the 1990s).

You could say "well, Russia and China could theoretically do the same". Sure, but that's the whole point, US kit is no more trustworthy than Chinese kit. Back doors in Chinese kit may at present be a theoretical possibility, but we know that US kit is being back-doored on a mass scale. If people just continue to stick their heads in the sand over US kit, then they may as well just set their root passwords to "passw0rd" and be done with it.

The solution is to trust no single outside party. That means making everything as open as possible, which provides fewer corners to hid back doors in.

It might have something to do with using keys that are too short. There have been notifications that the older 32 bit key ids are no longer secure and that users should be using longer ones.

There are also reports around that something like 95% of Blackberry users use short passwords which can be brute-forced in practical periods of time.

Both of the above are related to the fact that passwords or keys that might have provided reasonable security 10 or 15 years ago are no longer adequate because of the availability of cheap computing power.

Of course a third possibility is that they are simply going around the encryption in a poorly implemented third party app. If the app is keeping plain text draft, display, cached copies, or fragments of text from deleted memory, then the Dutch don't need to actually crack the encryption in order to get at least some of the messages.

There are also reports that people have been removing (de-soldering) memory chips from the Blackberry PC boards, or using the JTAG port to get a copy of the password hash, and then brute forcing it.

The thing which suggests that it is something to do with one of the above, or something similar, is that the Dutch were able to crack some messages but not others.

Cellebrite says their UFED analyser will: "Recover a greater amount of deleted data from unallocated space in the device's flash memory ... Rich set of data: Apps data, passwords, emails, call history, SMS, contacts, calendar, media files, location information etc. ... Decoding of JTAG physical extractions of a rich set of data from Windows Phone 8, BlackBerry 10, Android devices and more ... Recover deleted image files and fragments when only remnants are available"

If you download and read their supported devices list, they "support" a long list of Blackberry phones (and pretty much everyone else as well). I suspect the Dutch used a Cellebrite analyser to get actual data, and then applied some techniques of their own to brute force weak passwords including passwords which were re-used for multiple purposes. If the owner used the same passwords over and over again for different purposes (as many people tend to do), the Dutch only needed to find one app that had a weak point in order to get access to the rest.

Re: Boring technical details

Here's the figures for 2014: 45% nuclear, 6% hydro-electric, and the rest fossil fuel - so coal and gas. In other words, half of electric power generation capacity can be affected by coal or gas shortages.

https://www.eia.gov/beta/international/analysis.cfm?iso=UKR

More than 90% of Ukraine's coal comes from the Donbass, which is where the war is.

https://en.wikipedia.org/wiki/Coal_in_Ukraine

In 2014, 75% of Ukrainian coal production was for power generation. Coal production has plummeted, and Ukraine has had to import coal from Russia and South Africa.

https://www.kyivpost.com/content/ukraine/coal-output-in-ukraine-declines-224-in-2014-376952.html

So, if no coal or gas then the lights go out. Both have to be imported using scarce foreign exchange. Hence, my comment about unfortunate Ukrainians may be facing a cold, dark winter.

Re: A shortsighted view

@sabroni - "Funny how the reasoned, sensible post in the middle has a name attached. Getting of sick of people using Anonymous posts as an excuse to behave like dicks."

If these people who post anonymously are so confident of what they claim, why are they afraid to attach their user name to it? One might suspect it's because that even they know that what they're saying is complete bollocks and they don't want that history accruing to their user name.

Let's look at the post which started this particular thread. It's a vague broad opinion without any real substance. It's a marketing message in a comment along the lines of "now PowerShell gets your clothes 26% whiter than the leading brand!"

Anonymous comments on El Reg seem to come in three main flavours. The first is when someone is offering genuine inside information on an IT cock-up and they are afraid of repercussions should that comment be traced to them. Those types of anonymous posts are valuable, but very rare. They are however the main reason why the anonymous feature is there.

The second type is where some marketroid posts horse shit about how wonderful his product is. These are the most common anonymous posts. Based on writing style I suspect that most of ones that I've read come from just one salesman from either Microsoft or from a Microsoft sales partner who seems to have a lot of spare time on his hands.

The third type are troll posts. They post utter bullshit because they know that they'll get other posters all in a lather to refute the blatant falsity in the post. Troll posts generally try to be as vague as possible in order to avoid giving respondents anything solid to respond to and easily refute. Something like "x doesn't even come close to y" without any sort of criteria or supporting facts is a classic troll.

I suspect that the first post is the third type. It's a troll.

The best response to marketroid and troll posts is to treat them as such, and not as serious posts.

Re: It's easy to see how someone got mixed up.

"Treating "sessionAffinity" as being the same as "SessionAffinity" would've fixed that."

Actually, it wouldn't, because this is just one example of a misspell. It could just as well have been "sessionAffiinity". You've only solved a very narrow problem without really solving the problem in general. The only real solution is testing to see if something actually works.

"Remember back in the day when we used prefixes to indicate types of things? Can anyone remind me why that was such a bad idea?"

It was a bad idea because it added a lot of cryptic prefixes for no real good reason. If you changed the type, you had to change the variable or constant names. And what about when you defined your own types? Do you create your own prefixes? And what about when everything became an object? Do you prefix almost every variable with something that means "object". How was that supposed to be useful? And how would it work with dynamic languages where the base type of the reference can potentially change?

It really wasn't very useful. That's why it never really caught on outside of a fairly narrow niche of developers.

"I think Torvalds has won, hasn't he?"

Yes he has, and we can tell that because the Microsoft Windows sales account managers seem to have nothing to do these days other than post self-delusional anonymous career path self-justifications on El Reg about how they've still "got it" while sobbing into their beer in a lonely corner of a pub.

Nobody believes the crap they post, but as they click the "post anonymously" box on the El Reg comment form, they can tell themselves that they "stuck it to the man" as their career gets steamrollered beneath the flippers of relentless progress.

@Lee D - Apple is largely making the point that they are pushing their changes back upstream to the original projects rather than maintaining a separate fork with extra proprietary bits. Taking an MIT/BSD licensed project proprietary is something they could do if they wanted to, it's only GPL style licenses which require publishing the changes.

Perhaps you weren't involved in the open source world at the time, but one of the most significant open source projects they used was the GCC compiler system, which was the basis of not only their C and C++ needs, but also Objective C was based on it. Next (before their merger with Apple) tried to do a proprietary fork (not publish their changes). However GCC is GPL licensed. It took the FSF saying they would not allow this (they consulted a lawyer, who said that under copyright law the angle that Apple was trying to use to get around the license would be considered a "subterfuge", and a judge would come down on Apple like a ton of bricks) to get Apple to publish the source. Following the Next merger however, Apple complied with the license in an very passive-aggressive manner, meeting the letter of the law while being as much of a knob end about it as possible. When LLVM became viable (it's still not quite up to GCC when it comes to speed though), they switched to that for their compiler back-end because the license on that is similar to BSD/MIT, which lets them do things like say make Swift closed source unless they felt they could derive some advantage from making it open source. Apple also initially took the passive-agressive approach to license compliance with KHTML (or "Webkit" as Apple likes to call it).

Apple took a major black eye in the open source community from all that, so at every opportunity their PR department like to spout about what good open source community members they are because they use open source, etc., etc. The "contribute back" is emphasised because their allergic reaction to any sort of license which requires them to contribute back (e.g. GPL) makes people suspicious as to the depth of Apple's commitment.

Of course while Apple was going around scooping up open source software (instead of buying an expensive license from AT&T) and packaging it with their product, Microsoft was going around saying that open source was "cancer" and that company directors would be going to prison if they used it (MS constructed an elaborate legal theory that claimed that using open source was a form of fraud under US securities laws). MS were also quietly channelling shed loads of money to SCO to go around and sue world plus dog.

So while Apple isn't exactly the cuddliest company around so far as open source was concerned, they weren't the worst. And of course the Apple reality distortion sphere has fan boys proclaiming that Apple creates everything they touch.

Re: Not backwards compatible can cause a lot of problems

@DrXym - "It should have had 5 years of bug fixes and then frozen"

The LTS decision came about because of a lot of user demand, especially as it is an important part of most major Linux distros and companies like Red Hat have long term support requirements for it due to their own customer commitments. The core developers would rather have stopped support for 2.7 as soon as 3.1 came out. Having said that, there are no new features going into 2.7, just security fixes. All the major Linux distros however are in the process of switching to 3.x, or have switched already. Version 2.7 however isn't going to totally disappear until the last Red Hat customer still running it upgrades to a RHEL version that doesn't have it, some time in the 2020s.

@DrXym - "As for Unicode, from experience the best solution for any language is that all source code is treated as UTF-8."

Python has been around for quite a long time, longer than Java. Going to unicode was always going to be a problem, and the 3.x series is where they finally decided to bite the bullet and do it. The other changes came along for the ride, as they decided if they were going to make any breaking changes, they may as well do them all at once and get them over with. They weren't going to hack these changes onto the side, they made them thorough going and comprehensive to ensure they were done right and also done once and for all.

In 3.x, source is treated as UTF-8. With 2.x you could specify that at the beginning of the file, but it wasn't by default and most 2.x programs don't do it.

@DrXym - "At runtime the choice is harder since Unicode requires 32-bits encapsulate every code point but that's horribly inefficient.Java (and QT and Windows API) uses UTF-16 which *mostly* works until you hit some esoteric ancient language and then its guaranteed to break."

In the newer versions of 3.x, they encode UTF-8 string data using 8 bits, but switch to larger sizes if non-ASCII code points are encountered. This saves a lot of RAM in applications which process string data, such as web apps. Increased RAM consumption due to Unicode was in fact another factor keeping some people from switching to 3.x until that change was made.

The variable character length isn't a problem because that is all handled by Python behind the scenes. You don't get access to the raw memory buffers, you do it through Python's built in string methods and syntax. All strings are unicode, and binary data is now a different data type. There is no more mixing the two together.

Re: PHP et al

Very often what's in the introductory books is the wrong way to do it. The introductory books tend to be written to let students get something working as quickly as possible. Once you've got the basics of PHP down, you need to read some up to date more advanced books to find the correct way to do things.