back to article Apple Macs, iThings, smart watches choke on tiny Indian delicacy

Apple last month fixed a flaw in macOS and iOS that allowed a text message to crash its chat software (CVE-2018-4100) – and now it has the opportunity to do so again. Various macOS, iOS, and watchOS apps that rely on Apples's text-rending code can be crashed when sent a message containing a symbol composed of characters used …

  1. keithzg
    Facepalm

    Broadly speaking, there are two questions to ask when a bug is found in software:

    (1) What are the specific parameters of this bug?

    (2) Why is this bug able to exist in the first place?

    Obviously, #2 is broad and philosophical, and you can waste a lot of time and effort on it even before it comes to the potentially large time and effort it'll fix to address it meaningfully with actual code changes. But if you only ever ask question #1, you'll often only fix symptoms of fundamental problems in the approaches used; conversely, if you ask (and can find a real answer to) question #2, you can prevent future bugs and fix ones you don't even know about yet. It may cost you time and effort in the short term, but it can potentially save a LOT of time and effort in the long term.

    Apple seems to do a really bad job at approaching things from a perspective of fixing fundamental problems rather than just putting out fires as they appear, particularly considering the ungodly amount of money they have and thus potential resources to throw at such concerns. I mean, when the problem *last* month came up I went "haha, really, again?!" because *that* was far from the first time essentially the same problem has come up across their software.

    1. Anonymous Coward
      Anonymous Coward

      Perfect example of why the re-focus on stability/performance for iOS 12 was necessary

      In the last couple iOS releases I've noticed a couple glitches in animation every once in a while. Not often, maybe once a week, but I'll be swiping up or across and there will be a 'hitch' for a split second. Never used to see stuff like that before.

      There's NO WAY that Apple engineers haven't seen those, but lacking a Steve Jobs coming down and screaming at them that those sorts of bugs are not acceptable they seem to get overlooked. Say what you want about Jobs, but he didn't accept imperfection and wouldn't allow those who worked for him to accept it either. Hopefully they will get back that attitude, that's one of the reasons I've owned iPhones since 2009.

      1. Naselus

        Re: Perfect example of why the re-focus on stability/performance for iOS 12 was necessary

        "Hopefully they will get back that attitude, that's one of the reasons I've owned iPhones since 2009."

        I don't see why they would, tbh. Jobs was obsessive about his vision - he wasn't much of an engineer, and didn't really understand the tech, but he knew what it ought to be like from a user point of view. And he would scream blue murder at engineers for failing to deliver what he had in mind.

        Cook simply isn't like that. I'm not entirely convinced he has anything much in mind when it comes to a new product, aside from possibly just a string of dollar signs; he seems to have very little in the way of taste when it comes to tech products, and very little ability to judge what's a good idea and what's a non-starter. His hit rate on new product lines is basically zero, limited to the most raving fanbois only, while the company coasts along on iterating the iPhone ad infinitum.

        Seems to me, as long as Cook's in charge, there's no incentive for Apple engineers to look for perfectionism, since the top isn't interested either.

        1. Anonymous Coward
          Anonymous Coward

          Re: Perfect example of why the re-focus on stability/performance for iOS 12 was necessary

          Cook doesn't have to be the one personally driving the engineers like Jobs was, it can be someone a little lower on the totem pole. Cook is an operations guy, he should stick with what he knows. Its really Craig Federighi's job, but if he doesn't have that obsessive compulsive instinct for perfection he'll need to find a lieutenant who does. Doesn't have to be a screamer like Jobs, but someone who is willing to keep pushing back and making the engineers fix things until he's satisfied.

    2. Notas Badoff

      Compose yourself

      Key to this is that this isn't just one Unicode character, but a composed character. As the article says

      The symbol represents a combination of Telugu letters and signs, specifically the letter "ja," the sign "virama," the letter "nya," a zero-width non-joiner and the vowel sign "aa."

      Dumped as UTF-16 that's 0x0c1c 0x0c4d 0x0c1e 0x200c 0x0c3e

      So on-the-fly the font compositor has to figure out how to smash all these individual parts together into something that appears to be the one 'character' జ్ఞ‌ా . So stick the main part on the left, stick another part up top right, one more part swimming underneath, and the final part trailing behind on the right side, but the ZWNJ says keep it together with the first three smashed together pieces. on-the-fly!

      Now people like Behdad Esfahbod work on opensource like harfbuzz and pango that gets used by world plus dog, but 'world' doesn't always keep up-to-date with fixes. Nor necessarily use the APIs correctly. I'm not surprised there are combinations as yet untested within some applications by some vendors.

      Crashes on complex character compositions? Sure! Heck, entire countries meltdown using plain ASCII characters like T I B E T !

      1. Bandikoto

        Re: Compose yourself

        Worse yet, the country that melts down on the ASCII sequence T I B E T is right in the middle. 中

    3. Jim Mitchell

      (Q3) How did our QA process allow this bug to escape?

    4. Timmy B

      "(2) Why is this bug able to exist in the first place?"

      We would use the phrase "How would we prevent this happening again?". If you ask how it happened originally you may get an answer as vague and unsolvable as human error. If you look for way to mitigate and prevent then you may even stop that as much as possible. For instance we have rules where variables have sensible names above a certain length limit - this limits typos and the effect thereof.

  2. Anonymous Coward
    Trollface

    Wouldn't it be fun if someone posted this...

    On Trump's Twitter feed ?

    జ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ాజ్ఞ‌ా

    1. DontFeedTheTrolls
      Black Helicopters

      Re: Wouldn't it be fun if someone posted this...

      See icon

  3. Anonymous Coward
    Anonymous Coward

    I'm just here to say Sinofsky is a pillock

  4. Anonymous Coward
    Anonymous Coward

    OK it complex to render

    but why crash, surely the rendering function has a way of conceding defeat on things it can't manage in a more clean way. In the olden days you just got a rectangle box for unrenderable characters.

    It seems a slightly odd behaviour whereas I would have expected the character simply to render incorrectly.

    1. handleoclast

      Re: OK it complex to render

      but why crash, surely the rendering function has a way of conceding defeat on things it can't manage in a more clean way.

      Only if the coder anticipated the problem.

      In the olden days you just got a rectangle box for unrenderable characters.

      And still do, when the renderer recognises that it does not have a glyph for that character. Known problem, known solution: "I don't know what that character is supposed to look like so I'll draw a rectangle."

      This is a case of "take this character and that character and replace them with a single glyph of conjoined characters, then take another character and shove it under those combined characters, then..." except there's a bug in the code for doing that mashing and smashing. The code handles simpler cases correctly but falls over on the really complicated ones.

      Notas Badoff decomposed the glyph as:

      the letter "ja," the sign "virama," the letter "nya," a zero-width non-joiner and the vowel sign "aa."

      That's జ, జ్, ఞ, zero-width non-joiner, and ఆ (assuming my character mapper wasn't lying to me and I drove it correctly). When put together according to unicode specs, that should result in జ్ఞ‌ా. Which I think means "dog made by twisting long, thin balloons together if you don't know what a dog really looks like."

      It's a bug. What you're asking for is that coders put in code to handle bugs that they don't know are in their code. Which is like solving the halting problem, except not as sensible.

      1. Lee D Silver badge

        Re: OK it complex to render

        I don't know about the OP but what I'm asking for is for a font renderer to be able to either render a glyph or error out sensibly, not just crash. A crash means you didn't isolate your memory etc. from each other and you end up in a combination which causes things like divide by zeroes or out-of-range memory accesses or pointer confusion.

        Which, sorry, but shouldn't happen in production code that goes out to positively millions of devices. I can quite understand "this glyph is unrenderable, or something went wrong, or I got an error, so I caught that exception that the underlying code threw and returned NULL glyph in that one's place, rendered the rest of the string and then returned the whole - valid and renderable, just not correct - string to the underlying process without the potential for code execution or further errors propagating down".

        Seriously, people, this is what EXCEPTIONS and error-handlers are for. It means that even if the glyph-renderer throws an exception, the string renderer that called it should do something about that beyond just throwing that hot potato all the way back up the chain to the UI framework and then to an app that can't handle it causing the OS to kill the app. It should be replaced with a safe glyph, the string should be caught and marked as erroneous by the string rendering routines (and replaced with, say, XXXXXX), the UI framework should catch it, and app should BE ABLE to catch it if it propagates that far (but I don't expect it to).

        If you have not designed your UI framework to safely handle and isolate calling processes from errors in glyph rendering (which could be caused by a corrupt font, or an illegal combination of glyphs and modifiers, or a missing glyph from the font entirely etc.) then you shouldn't be writing one and FORCING people to use it (notice how ALL those apps are from different people, including the lockscreen, but they all are subject to the same textual API...).

        Throw... catch... it's not hard. And catch should never be "oh, just catch everything I might have forgotten and throw it back up the chain where nobody can do any better than I can anyway". That's the last resort.

        1. handleoclast

          Re: OK it complex to render

          I don't know about the OP but what I'm asking for is for a font renderer to be able to either render a glyph or error out sensibly, not just crash.

          I'd like all sorts of software to not crash. To be able to handle unexpected input and not crash. To be able to handle anything I throw at it and not crash. I'd like it, instead of crashing, to pop up an error box saying "This file is a corrupted heap of steaming shite" or "You just asked to do something incredibly stupid." This is the ideal behaviour. But Windows 10 is still out there.

          And yes, I'd kind of hope that a font rendering engine would handle both legitimate and illegitimate character sequences correctly. Or at least display an "I couldn't figure out how to display that" box. This is the ideal behaviour.

          Software isn't ideal. Sometimes people introduce bugs. Off-by-one errors. Walking off the ends of arrays. Stupid stuff like that. And it's not always easy to prove your code is correct or to test all possible cases. Which is what happened here. Some obscure character combination exposed a bug of some sort in the font renderer and it fell over.

          Yes, in an ideal world all software would have exception handlers that deal with bugs. Except for the ones it can't handle. Like when an exception isn't thrown. Like when the code does an array walk through all of the device's memory, or goes into an endless loop, or something like that.

          Yes, you can get around infinite loops with some sort of watchdog interrupt. Which can cause other problems, like false positives because your device is multi-tasking its way through too many monero-mining tabs in your browers. There comes a point where error-handling code becomes so complex it introduces more errors than it catches.

          In an ideal world the font renderer wouldn't have crashed but would have displayed an error message. In an idealer world it wouldn't have had a bug in the first place. In the meantime, try to make your peace with reality because you'll be spending the rest of your life here.

          1. heyrick Silver badge

            Re: OK it complex to render

            "In an ideal world the font renderer wouldn't have crashed but would have displayed an error message."

            Why? What possible sensible useful (to the end user) error message is there?

            In an ideal world, the font renderer would not have crashed but instead would have noticed it got itself into a twist over that character, so it threw in the towel (or maybe the exception handler) and output a simple solitary question mark in place of the extremely complex glyph.

            I think everybody would be happy with a '?' lurking amongst the characters because they can, at least, read all of the other spaghetti hoops instead of, you know, nothing at all.

      2. JeffyPoooh
        Pint

        "Only if the coder anticipated the problem."

        handleoclast offered, "Only if the coder anticipated the problem."

        No. Keeping the code 'on the rails' is Computer Science 101.

        "Unexpected" problems:

        * Throw an error message and do something sensible, yes.

        * Crash-out, no. That's an automatic failing grade.

        All inputs should be handled, even if they 'fall through the table' and end up with default ERROR output.

        1. handleoclast

          Re: "Only if the coder anticipated the problem."

          @JeffyPoooh

          All inputs should be handled, even if they 'fall through the table' and end up with default ERROR output.

          That's in an ideal world. In an idealer world all inputs would be handled correctly. Did I ever suggest otherwise?

          The OP appeared to be having difficulty understanding why Apple couldn't have handled Unicode characters the way they were handled 10 years ago, before combining characters and the like were introduced. The only problem back then was somebody using a character your computer didn't have a glyph for, so you'd get a rectangle instead. But it's no longer that simple. Unicode now has all sorts of complexities, so it's no longer a simple matter of mapping from a character code to a glyph.

          As you and Lee pointed out, the font renderer should have invoked an error handler if it got into difficulties (did I suggest otherwise?). It didn't. Not catching the error was a bug. Having the error was a bug. And because they were bugs, they were unplanned and unexpected. Which is why the font renderer didn't display the rectangle the OP expected but instead crashed. Because Unicode isn't simple any more and because programmers fuck up, especially when they write code to handle complicated stuff.

          Maybe I should have taken greater pains to explain that Unicode isn't simple any more as well as the fact that programmers fuck up. I thought I'd made it sufficiently clear that handling a missing glyph by displaying a rectangle is fairly simple but rendering modern Unicode is quite complex.

          1. Brewster's Angle Grinder Silver badge

            Re: "Only if the coder anticipated the problem."

            But this is fundamentally a case of recognising a sequence of code points ([U+0c1c, U+0c4d, U+0c1e, U+200c, U+0c3e]) and finding the relevant glyph in the font table. Yes, there are all sorts of combing characters that can cause multiple code points to render as a single glyph. But it shouldn't have to dynamically create glyphs, just find the right one.

            And my betting would be a mismatched set of lengths. U+200c (the zero-width non-joiner) normally forces two glyphs where one would otherwise be. But in this case it seems not to do that. So I'm guessing the text processing code naively allocates space for two glyphs but ends up outputting one. If the glyph-list is an array of pointers (into, say, the font table), then the renderer is going to crash when it hits a null or uninitialized pointer.

            Interestingly, when I pasted the character into the Chrome console, it displayed as two characters separated by a red dot, only converting to the final glyph as I hit enter. Chrome then uses a single glyph. By contrast, Firefox appears to use two glyphs to represent this character. So this is a confusing character. But everybody's behaviour is better than crashing.

  5. Anonymous Coward
    Anonymous Coward

    "so much for so long with such a high level of consistency"

    Said the man who killed Windows desktop consistency trying to enforce an UI which wasn't suited for desktop systems and servers at all.... what a jerk.

  6. Brewster's Angle Grinder Silver badge

    I'm guessing this thread doesn't contain many contributions from people with Apple hardware.

    జ్ఞ‌ా

    1. Mike Moyle

      Re: I'm guessing this thread doesn't contain many contributions from people with Apple hardware.

      FWIW -- I had surgery earlier today and am on painkillers, which may explain why I was loopy enough to open this in Safari on my IPad Pro (IOS 11.2.2). If all these posted glyphs have been generated correctly, then -- at least under the conditions listed above -- the bug apparently doesn't happen in Safari. The article DOES say that Apple's chat app is affected, but maybe people don't generally use that to comment on El Reg. ;-)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon