The Exiles almost clung on after Tristan Abrahams' fifth-minute strike, but even the most heroic defending couldn't prevent Jonjo Shelvey's brilliant late equaliser.
With Premier League regulars such as Shelvey, Ryan Fraser, Callum Wilson and Joelinton all on the pitch, it had looked like Steve Bruce's side would fail to score against their third-tier hosts and slip up.
But Shelvey broke Newport hearts in the 87th minute with a magnificent curling effort into the top corner.
Newport were forced to throw bodies on the line in the dying seconds and got the penalty shootout their performance deserved.
Exiles goalkeeper Nick Townsend kept out Joelinton, but Mark Gillespie saved from Ryan Taylor to level it up and take the shootout to sudden death.
And Brandon Cooper, who was exceptional throughout, became the unfortunate man to miss by blazing the seventh penalty high over the bar.
It means the Welsh side narrowly miss pulling off a seismic upset for the third year running, having dumped Leeds out of the FA Cup in 2018, and Leicester in 2019.
Newcastle, by the skin of their teeth, go through to the quarter-finals.
Newport were denied another stunning upset by a late Newcastle fightback as the Premier League club scraped past the League Two minnows on penalties.