I highly recommend making a visit to the Church History Library AND it's a great addition to a temple trip. The front doors lead you to a lobby complete with an information desk and seating. If you go beyond the lobby you will enter the area filled with tables, computers stalls, check out counters and aisles and aisles of books that you can look at. There is a second and maybe even a third floor that is filled with books, photographs, and microfiche that you can ask to look at, but you are not permitted to go up to those floors - at least I got the impression that the general public wasn't admitted upstairs. Another area on the main floor is the reading room. I think this is my favorite place in the entire library.
You have the option of taking a 15-20 minute tour of the building which I did. The missionary that led us on our tour didn't have a very loud voice and that mixed with the large building made it so that I couldn't hear everything she said and any additional noise made by anyone else made it impossible to hear her. Due to this and a short video also shown during the tour I would not recommend taking little kids (6 and under) on the tour if you are by yourself and your children have short attention spans. But that's me. Just thought I'd throw that in there since I was wondering myself.
I'm hoping to actually go in and access their material soon. I told BS that I want to go on a date w/ him there. I don't think he's too interested, but maybe some PF Chang's would make it worth his while.