From May to July 1999, Pakistan and India took up arms once again in what became known as the Kargil Conflict in the Kashmir area along the northern borders of India and Pakistan. The operation was planned and executed while Musharraf was Army chief of staff under Prime Minister Sharif. Kashmir militants with assistance from Pakistani soldiers took positions in Indian territory. They were soon discovered by the Indian army. Some reports indicate the Indian intelligence knew of their intentions weeks before the conflict. With the use of heavy artillery and night raids, the Indians slowly pushed back the militants and the Pakistani forces. The reversal was a complete blow to the Pakistani government, which had believed its forces had an advantage in the element of surprise. With Pakistani forces struggling in the field, national pride at stake, and many government officials beginning the blame game, the Pakistani army covertly planned a nuclear strike at India. But news of the plan reached U.S. President Bill Clinton, who gave Prime Minister Sharif a warning to stand down. Pakistan withdrew its forces, leaving the militants to be destroyed by the Indian army.

Prime Minister Sharif claimed Pervez Musharraf was solely responsible for the Kargil debacle while Musharraf claimed Sharif was to blame. In any case, the incident was a total embarrassment for Pakistan, not to mention a loss of prestige, morale, blood and treasure. On October 12, 1999, Sharif attempted to dismiss Musharraf from his position as commander-in-chief of the Army, but senior Army generals, loyal to Musharraf and believing the prime minister was distancing himself from any responsibility for the military defeat, refused to accept Musharraf's dismissal. Musharraf was out of the country, but when word reached him of Sharif's orders, he immediately boarded a commercial airliner for Pakistan. Sharif ordered the Karachi airport closed to prevent Musharraf's plane from landing. The generals seized control of Sharif's administration and placed Sharif under house arrest. He was later exiled to Saudi Arabia. Musharraf arrived at the capital and took control of the government. The sitting president of Pakistan, Rafiq Tarar, remained in office until June 2001, at which time Musharraf formally appointed himself president.