The E5420 has a plastic wrapped battery holder which has a 2 pin connector connecting onto a motherboard. To replace this, you'll either need to cut back the plastic, pull off the pads and solder in a new battery (CR2032 I would guess) or order a new CMOS for these which is dirt cheap since pretty much every machine using that style of battery uses the same 2 pin connector. The connector on this machine seems to be located on the BOTTOM of the motherboard, but rather stupidly shrouded by plastic meaning you're going to need to most likely get the entire bottom panel off - this could be as simple as just removing the rest of the bottom cover, or as complicated as having to have the entire palmrest off and motherboard out (so basically the entire machine in pieces) however I am unsure as I do not own one.
Chances are with the battery replacement causing it, it's either really good timing on the CMOS dying too, or more likely the CMOS has been dead for a while and your new battery does not for some reason have the ability of sustaining the BIOS settings (which is more likely if it's aftermarket and not Dell).
Here is the picture of a motherboard on these, notice the battery itself if located to the RIGHT of the CPU socket and to the LEFT of the USB port which would usually be on the LEFT side of the machine at the FRONT during normal operation.
This is the sort of replacement battery you will need, colour of it doesn't really matter. At the end of the day it is just a glorified CR2032 battery which has been soldered between two pads, with a plastic cover and a connector on the end. This is basically just a way of saving space on the board.
This should solve your issue, and if not try stuff like a BIOS update, your old battery back in, etc.