It turns out that the problem is that the spring loaded contacts that provide battery power to the main board become weak and no longer contact the board. This means that the device can lose power when bumped. To fix this I could either wire the battery directly to the main board or add a spacer to increase the pressure on the connectors. I decided to add a spacer since it is the easier solution.
The first and most difficult step in fixing the Edge is getting it apart. To do this, I used the back of a small knife to carefully scrape all of the glue out of the seam between the top and bottom of the case. The seam is the line between the black back of the case and the part of the case between the buttons. Don't try to separate the buttons from the gray part of the case. See the above links for pics of the seam.
It took at least half an hour to remove the glue - long enough that I started to have doubts that I was doing it right. Once enough glue was removed it almost fell apart. Don't pry apart the case without removing the glue!
Once the case was open I had a new problem. I needed to unscrew the two small Phillips head screws that secure the small board (with the USB port). Unfortunately I only had a #0 Phillips and it was too big. I bought a Harbor Freight 33 Piece Precision Screwdriver Set only to find the #00 and #000 wouldn't fit quite right either. I ended up using the smallest flat blade screwdriver bit and it came apart easily.
Once the small board was detached from the case I pulled the rubber grommet off of the USB port. I traced its shape onto a piece of old tube and cut the tracing out with a razor. An X-acto knife would have been useful here but it doesn't have to be that precise anyway.
I placed the new spacer onto the USB port followed by the original one. The original has a slot for the case built into it so I wanted those to line up. I didn't crank the screws all the way down because it seemed like it was deforming the board. Ideally the new spacer probably would have been a little thinner but I didn't have any other material.
I next cleaned the contacts with electronics cleaner. Then I applied silicone adhesive to the seam and reassembled the case. To hold it together I stacked some books on it over night.
After I was sure the glue was dried I gave the Edge a few test rides and it is fixed with no battery bounce. This project was a success!