nav search
Data Centre Software Security DevOps Business Personal Tech Science Emergent Tech Bootnotes
BOFH
Lectures

back to article
Check your repos... Crypto-coin-stealing code sneaks into fairly popular NPM lib (2m downloads per week)

JLV
Silver badge

Ouch. Clever, and simple, hack. Social engineering FTW.

How about an optional mechanism where an established github dev can be shown to have vouched for a new maintainer? A la web of trust?

Tons of holes, yes, but better than scammers just freely trawling the owners of no-longer-maintained popular repos.

Adam 1
Silver badge

Web of Trust, you mean that plugin that was caught with its fingers in the cookie jar a few years back? Maybe not.

Also, I wouldn't be so quick to assume that the new maintainer had a clue that this malware had been introduced. It is of course possible that s/he was in cahoots with the other account, but with the way that NPM works, half the web can break because some random Dev throws their toys out of the cot. Leaving asides the question as to whether his actions were justified, it showed that thousands (and that isa generously small number) of projects find themselves with unrealised dependencies.

This is both the greatest strength and Achilles heel of npm.

asdf
Silver badge

Build time internet dependencies are garbage

Guess I am old but I can't see ever supporting a code base whose build system relies on the internet at build time. 3rd party libs and external dependencies are going to be managed in-house only if its my butt on the line. Others outside your repo should not be able to break your builds. Granted would see same problem when you bring in broken code yourself but at least you can easily roll back and not depend on others to fix your build.

bombastic bob
Silver badge
Stop

Re: Build time internet dependencies are garbage

"Others outside your repo should not be able to break your builds."

ack

From the article: "This vandalism is a stark reminder of the dangers of relying on deep and complex webs of dependencies in software"

This is reason to STOP THIS PRACTICE FORTHWITH! (see icon)

'Teh Cloud' is WAY overrated here. More like "underestimated" [with respect to the damage it can do].

I know that _I_ do not want to be the mid-level software guy being phoned up at zero-dark-thirty because some _IDIOT_ 'chose poorly' and updated a Node.JS dependency. Blame goes on the one at the other end of the phone. "Not my fault" won't fly, either. B.S. rolls down hill, and now it's hit the fan!

[it's also why I won't use shared runtime libs with windows applications - static link or not at all!!!]

asdf
Silver badge
Trollface

Re: Build time internet dependencies are garbage

But then these kids couldn't use all these frameworks they hardly understand that makes them think they are programmers even though they need an external trim function written by someone else. They might actually have to gasp, learn how a computer actually works. Trolling now.

Adam 1
Silver badge

Re: Build time internet dependencies are garbage

> even though they need an external trim function written by someone else

Whilst I take your broader point, trim is exactly the sort of function that makes sense as a framework provided function. If you think it's trivial, then I'd chance that you really haven't understood its problem domain and what it counts as a character and what doesn't (and expectations in right to left languages) once you step outside the safe world of ANSI. And to implement it efficiently requires a good understanding of how your framework and runtime implement strings.

asdf
Silver badge

Re: Build time internet dependencies are garbage

Yeah I know but was trolling in general. Don't get me wrong the standard library and its ilk in different languages is usually a better idea than rolling your own for most things but some of these frameworks often seem used more as a faddish crutch than anything. You should at least understand somewhat what is going on under the hood which to be fair many people do.

GnuTzu
Bronze badge
Megaphone

What's the Count So Far?

It's NPM repos again. Anybody keeping track? Anybody got a fix? Sorry Node.js devs. I like Node.js too, but there's got to be room for improvement.

JLV
Silver badge

Re: What's the Count So Far?

its not node.js specific. could happen with any github-sourced code. in many cases the main code doesn’t even need altering, the underlying package install scripts (never looked at) can do the dirty. and those are not infrequently run under sudo.

In Python land, for example, that’s setup.py. Npm has active script install hooks too

:-(

GnuTzu
Bronze badge

Re: What's the Count So Far?

"its not node.js specific."

Oh, I wholeheartedly agree. And, that makes more work for those of us who have to work as gatekeepers. But, rest assured; we're keeping an eye out for these things.

overunder

Re: What's the Count So Far?

"its not node.js specific"

But, when in Rome. There is an allure to node.js when it comes to things like these. It's not hard to see why, node.js really is dependencie hell. I like node.js, but it's a toy with ever changing parts and never ending dependencies... it's fun that way.

Dr Paul Taylor

Javascript

Why did we ever allow this cr@p on our webpages in the first place.

I was just checking out the "more computing power than Apollo 11" adage. In fact your bluetooth devices have more computing power than Apollo 11 had. This seems to be the best account.

So, FIFY, it now takes more computing power to load one poxy webpage than the whole world had when Neil Armstrong walked on the Moon.

DougS
Silver badge

Re: Javascript

Much as I dislike Javascript, how is this specific to Javascript? This could have just as easily been an obscure Perl module or C library that is a dependency for something widely used. If updates can be pushed to those obscure modules/libraries with little or no oversight, and people get those updates when they download the "widely used" thing this will be a problem.

Not every project has a Linus looking out for bad code, let alone code with evil intentions. Even Linus isn't checking every line of patches to drivers for every random USB device. He's trusting that his downstream maintainers do that.

Adrian 4
Silver badge
Facepalm

Re: Javascript

I'm not a node developer and don't want to be. But if I read this correctly, it appears that all the components are dragged in without any testing, on the assumption that changes made to them will always be to remove bugs while retaining perfect backwards compatibility (and trustworthiness).

Most people's experience says this is not true. Even good fixes generate problems. The only way to be sure something works is to either test it, or to use a level of quality control of the components that we are decades from achieving in even the best controlled environments. If ever.

Don't you remember dll-hell, and it's cousin (prior to reliable package management) rpm-hell ?

A reasonable system that used components sourced elsewhere would hash every component it referred to and refuse to run unless it matched the combined hashes used in testing. Yes, that would preclude picking up fixes and improvements made in the intervening period. Yes, that would be a good thing, even without malware.

What's that ? You don't test things any more ? Well .. you're screwed, then.

pertayter

Re: Javascript

Re DLL-hell RPM-hell

Somehow over the years, these were magically fixed

The question remains, what happened in this case in the npm repository?

Also, what do other open-source Git-controlled projects do to avoid similar problems

The only obvious process error is that the original developer handed the package to the malware developer

In other free software projects, the normal way a stranger takes control of an abandoned package is to fork a copy in his own repository. I sympathise with the original developer wanting to abandon his package, but he should have either deleted it (with consequences for dependent apps) or just stopped updating it

Your point about testing is valid, to a point

In this specific case, is it reasonable for a developer to anticipate the introduction of malware which leaks confidential keys to a thief, and test for it? Until the theft actually occurs, the app works perfectly in a normal testing scenario

Do any other Bitcoin wallet apps test for leakage of secret keys?

Adam 1
Silver badge

Re: Javascript

> The only obvious process error is that the original developer handed the package to the malware developer

Nope. I mean it isn't impossible but my money is on the new broom adding a dependency without considering the integrity of the flatmap-stream package.

MacroRodent
Silver badge

Re: Javascript

> Much as I dislike Javascript, how is this specific to Javascript? This could have just as easily been an obscure Perl module or C library that is a dependency for something widely used.

Could hit Perl, perhaps, but in the case of C (or C++) there is no tradition of automatically pulling dependencies off the web whenever you build. Your program typically depends only on libraries that either are part of the platform, or ones that you explicitly install beforehand. In the latter case they usually come from a vendor, or (typically in Linux or BSD) from the curated and digitally signed collection of packages the distribution maintains. These mechanisms make slipping bad code in harder (Not impossible, but much harder).

yoganmahew

Re: Javascript

@pertayter

"In this specific case, is it reasonable for a developer to anticipate the introduction of malware which leaks confidential keys to a thief, and test for it? Until the theft actually occurs, the app works perfectly in a normal testing scenario"

Well, a specific malware threat, maybe not reasonable to expect, but that unspecified malware can be introduced through a repo the developer has no control over? Absolutely reasonable. It has happened multiple times so it must now be considered a known risk and you should have mitigations in place.

Alan Bourke

Re: Javascript

yeeeeees ... but the Apollo computers were designed to do a few things perfectly, they weren't really general purpose.

2+2=5
Silver badge

Re: Javascript

@Adrian4

> What's that ? You don't test things any more ? Well .. you're screwed, then.

As a general rule you are exactly right. But, in this case regression testing wouldn't have highlighted the malware because it was so specific to one environment.

What might help is code coverage testing - but I don't develop in JS so have no idea if the tools exist or not. The idea being that maintainers run a code coverage test after the regression tests have run, with a view to getting a report on lines of code that weren't executed during the regression tests. This would show up additional 'lurking' code and, possibly, be configurable to show code that previously wasn't executed (e.g. functionality available in a library but not used) now being executed unexpectedly.

This requires code coverage tools to exist and a degree of skill in using them. Something that repository owners should work towards automating as much as possible, rather than relying on maintainers, I feel.

dajames
Silver badge

Re: Javascript

Could hit Perl, perhaps, but in the case of C (or C++) there is no tradition of automatically pulling dependencies off the web whenever you build.

... and even if that were the case a developer might be expected to do some testing and measurement of the program once it was built, and would stand some chance of noting that something was not right.

Because Javascript is interpreted (OK, JIT-compiled, let's not split hairs) the problem is even worse: a Javascript program running in a webpage may pull in dependencies from the web whenever it is run. There is no opportunity for the developer to notice any errant behaviour, and little chance that the user would recognize it until it bit him.

Jens Goerke
Coat

Testing

Why should we pay for testers when we have customers paying us?

This kind of mind(?)set seems to be taught in MBA courses nowadays.

Mine's the one with the undocumented Z80 opcode list in the pocket.

bombastic bob
Silver badge
Meh

Re: Javascript

"Why did we ever allow this cr@p on our webpages in the first place."

Or in the back-end of a server, for that matter (i.e. Node.js).

JQuery and Node.js - the MALIGNANT TUMORS of 'Teh Intarwebs'.

There are better ways of doing these things. And they don't come with the *KINDS* of problems we see with client-side scripting (viruses, tracking, side-channel attacks, bitcoin mining) and server-side scripting with Node.js (single dependency update creating fail or malware on MANY servers at once).

Time for some CHEMO-THERAPY I say. Kill the tumor BEFORE IT GETS BIGGER.

really_adf

Re: Javascript

... unspecified malware can be introduced through a repo the developer has no control over? Absolutely reasonable. It has happened multiple times so it must now be considered a known risk and you should have mitigations in place.

I think that's the point of version pinning mentioned in the article. This should force a change to a dependency to require manual intervention, thereby providing an opportunity for the change to be assessed to the desired level of detail.

But it's inherently manual so, in many cases, it won't be done at all, and in some, things will be missed.

Jamie Jones
Silver badge

Re: Testing

LD Ix, 45 !

Simon Brady

Where old software goes to die

Reading the comments on the GitHub issue, I can sympathise with Tarr saying "I no longer wish to be burdened with responsibility for this code, no matter how many people may have come to depend on it." Far too many of us who consume open source software feel entitled to upstream support even though that's not how the contract works (I'm guilty myself of muttering imprecations at authors of code which has cost me precisely nothing).

That said, fading support and ambiguous deprecation is a real problem, both with OSS and non-free products (non-free vendors seem a bit better at formally ending support so at least you know where you stand, but there are plenty of exceptions). If "the community" needs to maintain the integrity of an abandoned project, then we need a process for reliably doing that, but first we need clarity that the project has been abandoned - that includes walking the dependency tree to see if there's buried reliance on code that hasn't been touched in years.

Maybe that's a useful function of repositories like NPM, to help people assess the risk of using packages they host?

Anonymous Coward
Anonymous Coward

No software can be trusted

Open source can be maliciously modified.

Closed source can have malicious intent built in.

There is no such thing as trustworthy code.

A.P. Veening

Re: No software can be trusted

"There is no such thing as trustworthy code."

I beg to differ, I have a tendency to trust code I've written and tested myself. And yes, I do realize I have to trust the compiler for that, but that hasn't been a problem so far as the IBM compilers are pretty reliable. Unfortunately, the days of directly entering hex-code are long in the past for me.

wolfetone
Silver badge

Re: No software can be trusted

"I beg to differ, I have a tendency to trust code I've written and tested myself. "

But we can't trust your code, and that's what they were getting at.

David Bond

Re: No software can be trusted

Forgot one, commercial software has little financial incentive to include malicious intent.

Anonymous Coward
Anonymous Coward

Re: No software can be trusted

Exactly - so write your own damn code and stop trying to pinch someone else's code just so that you can get the job done, collect your paycheck and move to the next job.

When people pay peanuts for coders that it's not surprising that we see this sort of thing all the time - how many more coin/cc stealing code is out there, no way have we found it all.

bombastic bob
Silver badge
Devil

Re: No software can be trusted

"But we can't trust your code, and that's what they were getting at."

A reasonable compromise (what we've been doing all along except for Node.js):

a) open source

b) well-tested prior to release

c) well-defined source snapshot identifying the release version (or fork in the repo, depending)

d) lots and lots of peer review

e) wait until the dust settles before upgrading to 'latest version'

Seems to have worked for me outside of this insane method of 'dynamic continuous update to bleeding edge' method being used for Node.js .

Linus has managed this with Linux for a long time. Having such a project manager makes a BIG difference. Having an official test+release system (and actual QA) does, too.

desht

Re: No software can be trusted

"Forgot one, commercial software has little financial incentive to include malicious intent."

Shame Sony BMG didn't get that memo. Or Lenovo, for that matter.

Gene Cash
Silver badge

Debian vetting & trust

You don't see very many stories about malicious Debian packages.

Prospective Debian developers are vetted by "application managers" who are folks that have been Debian developers themselves for at least six months.

Also: "Every application to become a Debian Developer must be advocated by an existing Debian Developer who is familiar with the applicant's work, and who believes that he/she is of a suitable standard both technically and socially."

https://wiki.debian.org/DebianDeveloper

I don't see why this isn't an industry-wide practice.

Reaps

Re: Debian vetting & trust

"You don't see very many stories about malicious Debian packages."

That didn't work for fucking systemd, which idiot let that shit in..

Wellyboot
Silver badge

Re: Debian vetting & trust

@Gene - that system is called trust, it doesn't prevent the system being undermined over time by gradual infiltration.

Malicious gits have been misusing other peoples trust since we climbed down from the trees.

Chairman of the Bored
Silver badge

Re: Debian vetting & trust

Yes... And I'd also submit that audio under Linux has gone from 'usable' to 'confusing as hell' in the last eight years.

bombastic bob
Silver badge
Thumb Up

Re: Debian vetting & trust

"You don't see very many stories about malicious Debian packages."

YES!!!

'The bleeding edge' is highly overrated. Production servers need stability, not 'bleeding edge'.

bombastic bob
Silver badge
Meh

Re: Debian vetting & trust

"That didn't work for fucking systemd, which idiot let that shit in.."

Yeah, well, I use Devuan to avoid systemd, and it derives from Debian. This doesn't mean Debian's practice of "being stable" isn't a really good feature of their distro (or the ones derived from it, like Devuan). There's still a choice.

Peter Prof Fox

Towards a better library/app model

A lot of software in these repos are developed by a single person. Somehow they're expected to be coder, tester, librarian, packager, instant tech support, long term support and safely guide noobs to use their thing in the right way for a suitable purpose.

I suggest a better way to develop software is to have teams. Even when the local theatre puts on a 'one man show' there are techies, and front of house and marketing involved. The trouble is we don't have recognisable templates for such outfits that everyday coders, tech authors, librarians, testers etc. can use to club together.

Brewster's Angle Grinder
Silver badge

Re: Towards a better library/app model

"The trouble is we don't have recognisable templates for such outfits that everyday coders, tech authors, librarians, testers etc. can use to club together."

Because most of those jobs are deeply boring and people wouldn't do them unless paid.

John Styles

The rigour of it all

They don't call it Software Engineering for nothing.

DropBear
Silver badge
Facepalm

Re: The rigour of it all

Oh yes, particularly that "if builders built houses the way programmers built programs, the first woodpecker to come along would destroy civilization" bit - it applies just a well with engineers for builders.

pertayter

Re: The rigour of it all

This topic is much deeper than it seems

Computer programming is not like building or engineering, because it is not constrained by the physical realities of Newtonian mechanics

This lack of constraints leads to a SMOP belief. Can we add this that or other new feature? Yes, SMOP. It's simply a matter of programming

Then software complexity rapidly exceeds any human ability to keep track or control

With discipline - management and end users should avoid asking for new features, programmers should have the courage to refuse - software could be manageable

But we're only human

SMOP

Adrian 4
Silver badge

Re: The rigour of it all

Here's an argument from over 30 years ago on the problems of software complexity.

https://www.nytimes.com/1986/09/16/science/software-seen-as-obstacle-in-developing-star-wars.html

https://www.jstor.org/stable/40725085?seq=1#page_scan_tab_contents

I don't believe we've made significant progress since.

Omgwtfbbqtime
Silver badge
Holmes

developers have only themselves to blame for trusting code that comes with no guarantees.

I think that line says it all.

Anonymous Coward
Anonymous Coward

Broken

Open source is badly broken.

Adrian 4
Silver badge
Trollface

Re: Broken

Wrong icon. You wanted this one ->

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

The Register - Independent news and views for the tech community. Part of Situation Publishing