Kalsoom Nawaz managed to win the election from northeastern Lahore city -- a stronghold of her husband's Pakistan Muslim League (PML) -- after garnering over 59,000 votes.
She is currently being treated for cancer in the UK. Sharif's daughter, Maryam, led the campaign for the seat in Lahore.
Her rival candidate, Dr. Yasmin Rashid from Tehrik-e-Insaf (PTI) of Pakistan's former cricket star Imran Khan, obtained more than 46,000 votes, local Dunya TV reported, quoting unofficial results.
The seat had fallen vacant after the country's Supreme Court disqualified Sharif in whistleblower Panama Papers scandal on July 28.
In 2013 elections, the former premier Sharif had secured over 91,000 votes and won the election by a thumping lead of more than 40,000.
Railway Minister Khawaja Saad Rafiq said the people of Pakistan had rejected Sharif's disqualification by the Supreme Court by electing his wife.
The opposition candidate, for her part, accused the government of rigging by using state machinery.
Some analysts see the by-election results as "alarming" for the ruling party, which was expecting a huge victory margin following the former premier's ouster, which according to the PML supporters, had stoked a "sympathy wave" in favor of Sharif.
"These results show that the voters, in general, have not accepted Sharif's contention that he had been ousted under an orchestrated conspiracy," Kamran Khan, a Karachi-based political analyst, said.
Sharif, 67, had resigned after being disqualified. The court ruled that he had acted in an untrustworthy manner by failing to declare a salary from his son’s Dubai-based company ahead of the 2013 election.
The court had also ordered the opening of corruption cases against Sharif and family members over revelations stemming from the Panama Papers scandal.
Sharif, who has held office as prime minister on three occasions but never completed a full term, has already been summoned by an anti-corruption court in a corruption case on Sept. 19. However, the former premier who is currently in London to see his ailing wife is unlikely to appear before the court.