If you require hotel accommodation in the area then first try a hotel booking service provided by Football Hotels who are powered by Activehotels, who specialise in locating accommodation near or within a short travelling distance of the football ground.
They also have the added advantage over some other hotel booking services that you pay on departure.
To access their Millwall & general London page click here.