If you give the junk a nudge, the spacecraft that nudged it would also be in the same orbit as the junk.
To nudge the junk without following it, you would need some way to "fire" it away, but every action has an equal and opposite reaction, so you would still need to expend the same amount of delta-V to correct the orbit of the cleaner ship.
I always wondered if you could use a long cable of something (carbon nanotube?) and attach that to the space junk, then fire the other end of the cable into the earth's atmo, thus aerobraking the junk into the atmo. Maybe some kind of parachute type structure on the end to increase the friction? If the coil and chute were light, you would then minimize the amount of dV required to fire it into the atmo, and also the amount of dV required to correct the orbit of the cleaner ship. Hell, if you attached a little rocket to the chute end of the cable, and didn't fire it until you released the junk, it would mean that the cleaner ship's orbit was not affected at all.
I suppose though, that anything in a lower orbit could get clothes-lined by the cable ;)