Three goals in the space of 10 minutes either side of half-time saw Milivojevic put his team in front in stoppage-time, but a Granit Xhaka free-kick and a controversial goal by Pierre-Emerick Aubameyang saw the game completely switch.
However, with eight minutes to go Wilfried Zaha won another penalty for Palace, and the Serbian skipper slammed it into the net to seal a point for his team, who arguably deserved more from a thrilling encounter.
Despite one side coming into the game with 11 straight wins and the other not having scored at home in five games, the Eagles ignored the form book to enjoy the better start and they nearly took the lead twice around the 15-minute mark.
Firstly, Andros Townsend fizzed a pass across the area that evaded everyone and fell to Zaha who got the ball under control and thumped a shot off the base of the post, and 60 seconds later a brilliant Arsenal-esque passing move involving Jordan Ayew, Patrick van Aanholt and Zaha created a great opportunity for Townsend, who failed to net for the third time in four outings against the Gunners by dragging his shot wide.
After a slow start, the visitors began to move through the gears and Alexandre Lacazette fired wide after seizing possession on the edge of the box. Mesut Ozil then caused problems on the right before picking out Hector Bellerin in a good position but an excellent sliding block by Aaron Wan-Bissaka snuffed out the chance.
But Palace continued to hold their own and had a first strong shout for a penalty when Zaha played a no-look pass into Ayew’s path who raced in front of Shkodran Mustafi and legs tangled inside the area however Martin Atkinson waved play on, but in first half stoppage-time the German did concede a spot-kick that left the official in no doubt.
After a corner failed to be cleared, the ball came to Cheikhou Kouyaté who was rashly launched into from behind by Mustafi. Showing real guts after missing from 12 yards seven days earlier, Milivojevic confidently stepped up and drilled into the bottom corner of Bernd Leno’s goal to net Palace’s first goal in SE25 since May.
However, the Eagles’ season of misfortune continued as 10 minutes into the second half they saw the game turn on its head through little fault of their own. On 51 minutes the visitors earned a free-kick on the corner of the area and Xhaka whacked it towards the top corner. Despite Wayne Hennessey getting a fingertip to it, Palace’s woodwork luck failed them again as the ball kissed the top of the post and flicked into the net.
And four minutes later, controversy hung over Selhurst as a corner was swung into the box and Lacazette flicked it towards goal with his airbound hand, allowing Aubameyang the chance to squeeze the ball over the line from close range despite Hennessey providing heroics to attempt to keep it out, but the goalline decision system informed Atkinson the ball crossed the line by a millimetre or two.
Incredibly after two huge setbacks, luck continued to desert Palace and substitute Max Meyer was played in by Zaha and despite a tight angle, he was next to hit the post before blasting over in a congested box.
For all their positivity, the hosts scarcely deserved to be behind, but on 82 minutes they were handed a chance to level things up from the spot once again. On the counter, Alexander Sørloth charged forward and fed Zaha, who tricked his way into the box and was brought down by Xhaka.
Once again, Milivojevic took the responsibility upon his shoulders and dispatched the chance to score his fourth goal in his last three outings against the Gunners, but more importantly clinch a point for his team.
Palace: Hennessey, Wan-Bissaka, Tomkins, Sakho, Van Aanholt, McArthur (Meyer 61), Milivojevic, Kouyaté (Puncheon 88), Townsend, Ayew (Sørloth 64), Zaha.
Subs not used: Guaita, Kelly, Riedewald, Schlupp.
Arsenal: Leno, Bellerin (Lichtsteiner 45), Mustafi, Holding, Xhaka, Guendouzi, Torreira, Iwobi, Ozil (Welbeck 68), Aubameyang (Ramsey 78), Lacazette.
Subs not used: Cech, Sokratis, Smith-Rowe, Mkhitaryan.