It was a lively start to Palace's home game against West Ham United, with both sides having chances to take an early lead.
First off, Dimitri Payet failed to properly connect with a low Lanzini cross, scuffing his sliding effort wide from close range. Then Palace settled and began to fashion half-chances.
Gayle and Puncheon were both crowded out during a goalmouth scramble and the latter dragged an effort wide, as Palace spent the opening 10 minutes trying to split the Hammers' backline open.
It was the visitors who struck first, when Victor Moses played Carl Jenkinson through to prod home. However, 30 seconds later Gayle went up the other end and won the Eagles a penalty.
Yohan Cabaye coolly scored from the spot, but had to retake it after Gayle was seen running into the area before the midfielder connecting with the ball. Thankfully for Palace Cabaye made no mistake with the retake, though Adrian did get a hand to it.
The action continued through the first half, with both sides looking to take the lead, but neither could find the final touch and the drama took a different turn on 43 minutes, when Gayle received a second yellow card. The Eagles striker caught Kouyate late, having been booked earlier for a foul on Payet, and received his marching orders from Mark Clattenburg.
It was a less dramatic start to the second period, with Zaha being sacrificed for Bakary Sako, Palace had to soak up a lot of West Ham pressure, as the visitors looked to make their man advantage count.
The Hammers enjoyed the lion's share of possession, pinning Palace back, but there were few clear-cut chances for the visitors, as the Eagles stood firm. Despite the pressure, Palace still looked dangerous on the counter, with Bolasie and Sako's speed worrying the West am defence on a few occasions.
With less than five minutes to go, Palace continued to defend heroically. Ledley, Jedinak and Dann all made important blocks and clearances to keep the Hammers at bay, but with two minutes left of regulation time the visitors finally struck.
A deep cross found Andy Carroll at the back post and he was able to nod down to Nikica Jelavic, whose blocked shot fell to Manuel Lanzini and the Argentine was able to rifle home a dramatic late winner to keep West Ham's impressive away from going.
Payet made sure of the result in the final minute of the four added on, beating the offside trap to go one-on-one with Wayne Hennessey and chip beyond the Palace goalkeeper. The scoreline was not reflective of the Eagles' performance, with the second half seeing some outstanding defending from several Palace players.
Palace XI: Hennessey; Kelly, Dann, Hangeland, Souare; Cabaye (Ledley, 76), McArthur (Jedinak, 60); Zaha (Sako, 45), Puncheon, Bolasie; Gayle.
Subs: McCarthy, Delaney, Campbell, Bamford.
West Ham XI: Adrian; Jenkinson, Tomkins, Collins, Cresswell; Noble (Carroll, 60), Kouyate; Lanzini, Payet, Moses (Zarate, 74); Sakho (Jelavic, 85).
Subs: Randolph, Ogbonna, Obiang, Valencia, Carroll.