There are several reasons for this need to double check security credentials, but the major one is the number of Soviet spies uncovered after the Soviet Union disintegrated in 1991. Some agencies were found to be more susceptible to Soviet penetration (basically because of sloppiness), and all of a sudden, a security clearance didn't mean the same thing everywhere. That's because that, while all initial clearances are approved using basically the same background investigation techniques, as time goes on, the agency you work for is responsible for ensuring that you remain eligible for whatever clearance you have. The post-Cold War spy scandal (which uncovered people working for other countries as well, like China and Israel), made all agencies more aware of the need to keep an eye on their people, but particularly on new people who have been around for a while, but at some other agency.
All this is typical bureaucratic cover-your-ass behavior. Better to sit on something, than to take a chance, no matter how slight, of being embarrassed. While the media will never jump all over this sort of thing (too geeky and unexciting), people in the intel business (who are getting burned by the bureaucratic delays) have gotten through to some members of Congress, and heat is being applied. That won't necessarily solve the problem, because advancement in the intel business is more a matter of avoiding failure, than in achieving success. Victories remain secret, but failures often become headlines.
The recent reorganization of American intelligence agencies, and the creation of the post of Director of National Intelligence, has failed to eliminate a lot of friction between the various agencies. Case in point is the reciprocal recognition of security clearances between agencies. That is, if someone has a Top Secret clearance in the CIA, and moves over to work at the Department of Defense, that Top Secret clearance should be, well, a Top Secret clearance at the Department of Defense as well. But often, it isn't. For over a decade, the various intelligence agencies have been getting slower and slower in recognizing the validity of the clearance of the person coming in. It can take up to six months for such transfers to be approved. In the meantime, the transferred person cannot go near classified information. This wastes a lot of money, and delays essential projects.