There are three types of coolant cell, each used to craft the next.
When used in a Nuclear Reactor, a coolant cell serves simply as a large heat buffer. It does not cool itself, nor will it exchange heat with nearby components (on its own); it can be placed next to a fuel rod to absorb its heat, and/or have heat removed from it with heat exchangers, for example.
A damaged, or partially heated, coolant cell cannot be placed into a fluid reactor; however, an undamaged coolant cell may be used in one, and take in heat, and then have it removed with heat exchangers as usual. So long as the coolant cell is not removed, it can stay in the fluid reactor in its damaged state. If it is removed, however, it must be placed in a bare EU reactor (one without a pressure vessel) and cooled back to an undamaged state before it can be used in a fluid reactor again (meaning that the heat it would have generated from being cooled is wasted).
A fully damaged coolant cell, like most other components, will melt and disappear.
The only other use for coolant cells is crafting Overclocker Upgrades.