The Mingei is a folk art museum that has pieces from all over the world. Right now they have a Maneki Neko exhibit (Japanese Lucky Cat) that my family and I enjoyed. Unless you come on a family Sunday, where entrance is discounted and there are activities for kids to do, I think preschoolers won't last long here. My daughter liked the cats and I think our visit was no more than 30 minutes, which I consider a victory since she's 4. Older kids interested in art will enjoy it, I think.
It's located just a few minutes walk from the Natural History and Fleet Science museums. So you could actually take the kids to one of these first (or last) and then take a quick stroll through the Mingei. They have an amazing gift shop, actually. Unique crafts (even for kids), jewelry, books and Japanese tea sets.