Once "Caps Lock" key is set to "On", the "Caps Lock on" indicator is displayed on the desktop screen permanently.
When the Caps Lock key is set to "Off", the "Caps Lock off" indicator is displayed for a 3 seconds and disappears.
This seems to be the default setting for the "On-Screen Display" package that is part of the "Hotkeys Features Integration" package.
All ThinkPad systems running on Microsoft Windows 7, Windows Vista, Windows XP, Windows 2000 configured with "Hotkey Features Integration":
To remove the Caps Lock On indicator from displaying permanently on desktop screen.
The on-screen indictor functions are installed as part of the Hotkey Features Integration package and is available from Support Site, download the latest version.