The last time I open a 6v battery like that was ....sheegh..20 years ago... anyway it had 4 cardboard lined 'F' calls, and it was potted with 'tar' looking stuff. LM2K, it is safe to open the battery casing, as long as you dont use certain tools, like... drills, hacksaw, axe... ya know, stuff like that. The rectangle box is just that, a box for the 4 steroid injected 'D' cells.
Jeff