The decree said trials for abuses committed before 2003 would be halted after three years.
But Mr Gracia asked for it to be annulled hours after Mr Vargas Llosa called it an "amnesty in disguise".
Peru's army fought bitterly against leftist rebels in the 1980s and 1990s. The military has been accused of rights abuses, but few people have been charged. Critics argued that the decree in effect made it impossible to bring successful prosecutions.
Mr. Vargas Llosa, Peru's most famous novelists, wrote in an open letter to Mr Garcia that the measure "constitutes a barely disguised amnesty to benefit a good number of people condemned or being prosecuted for crimes against human rights - murders, tortures and disappearances".
In protest, Mr Vargas Llosa resigned as head of a commission to build a memorial to victims of Peru's conflict with the Shining Path Maoist guerrillas, which claimed almost 70,000 lives in the 1980s and early 1990s.
"There is, in my judgement, an essential incompatibility between, on the one hand, supporting the erection of a monument to the victims of the violence unleashed by the terrorism of Shining Path, and on the other, opening the back door of the prisons with legal trick," he wrote.
Mr. Vargas Llosa suggested Mr Garcia may have issued the decree to protect himself from any future prosecution over massacres committed by the armed forces during his first term in office in 1985-90.
Shortly after the letter was published, the president's office announced it was seeking an urgent annulment of the decree.
"To avoid the decree being put to bad use, the executive office deems it appropriate that it be revoked," officials said in a message on the social networking website Twitter.
Analysts say the decree may also have protected ex-President Alberto Fujimori - an old rival of Mr Vargas Llosa who is already in prison for human rights abuses.
The novelist was defeated by Mr Fujimori in the 1990 presidential election, and he has since spent most of his time outside Peru and has taken on Spanish citizenship.