A moment of in the game from Claude Davis proved the turning point in this&nbsp;fixture, as his second half dismissal allowed Ipswich to grab two quick fire goals to seal the three points.
The defender pulled back Ipswich striker Jon Stead in the box&nbsp;for a penalty on 49 minutes, which resulted in a red card for him and the chance for Grant Leadbitter to calmly place the ball past Speroni.
Their second came four minutes later when Leadbitter turned provider with a perfect pass across the six yard box for Carlos Edwards to smash home.
The numbers were levelled up 17 minutes from time when Andros Townsend saw red for a terrible challenge on Speroni, which allowed Palace the chance to come back into the game.
Hopes of a come back were momentarily lifted when Neil Danns scored three minutes into stoppage time and the Eagles piled on the pressure in the closing stages, but in the end the challenge proved too big and the visitors closed out the game for their second win of the season.
Going into the match, George Burley rang two changes from the side that had travelled to Oakwell last weekend,&nbsp;with Neil Danns and Julian Bennett brought in to replace Darren Ambrose and David Wright, who both suffered injuries in training on Friday.
On the bench were three of the club's latest signings, Alex Marrow, Jonathan Obika and Stuart O'Keefe, while Edgar Davids and Pablo Counago watched from the stands.
The game almost started in the best possible fashion when Owen Garvan was picked out on the edge of the box, flicked the ball up with his left foot before firing a looping effort that flew&nbsp;inches wide of&nbsp;Marton Fulop's left-hand post 17 seconds in.
In response, Jon Stead saw a curling left footed drive comfortably saved by Speroni, while Carlos Edwards should have put the visitors into the lead when he connected with a Stead cross four yards out, only to smash it over the bar on ten minutes.
Throughout the half Palace&nbsp;were patient in their build-up and keen to put together flowing passing moves to break through Ipswich's defence, but they struggled to find the killer touch.
On 18 minutes, Zaha, who had had a tough opening period of the game as the Ipswich defenders set about unsettling him at every opportunity, came to life with a moment of superb skill.
Picking the ball up on the right flank he squared up two defenders before flicking the ball between them, running on to the ball and playing a dangerous ball right across the six yard box.
Had someone been able to connect with it it would have been one of the goals to savour, but unfortunately it was just a little too quick for the three supporting Palace players.
Palace thought they had found their&nbsp;way through when&nbsp;Andy Dorman&nbsp;had the ball in the back of the net on 22 minutes.&nbsp;
The midfielder, who was filling in for Ambrose on the left, latched onto a first time Alan Lee pass, rounded Fulop before passing the ball into an empty net, only to look over and see the linesman's flag raised for offside.
Andros Townsend then saw his effort from 25 yards dragged wide of Speroni's left hand post as Ipswich came into the half more in the later stages, while the Eagles keeper had to pull off a fantastic save to keep them out on 31 minutes.
Speroni managed to get enough of a fingertip to touch the ball onto the bar, after Stead's cross deflected off Davis and looped goal bound. It proved the last chance of the half, as the fixture set itself up for an intriguing conclusion going into the break.
It was disaster early in the second period though when Claude Davis pull back Stead, who went over rather theatrically.&nbsp;The referee pointed to the spot and Davis was shown a red card for being the last defender. Grant Leadbitter stepped up to take it,&nbsp;and sent Speroni the wrong way to give Ipswich the lead.
It was&nbsp;2-0 shortly after when Ipswich used the one-man advantage to good effect with a quick break, and Edwards smashed home Leadbitter's low cross into the six yard box.
It should have been three&nbsp;on 65 minutes though when Gareth McAuley had a free header in the box from a quick corner, but he angled it wide of Speroni's right hand post.
Unfortunately the game descended into scrappy period fuelled by some overly physical challenges, as Ipswich clearly&nbsp;set about aggravating the Palace players at every opportunity.
In the end it resulted&nbsp;with another red card, this time for Ipswich,&nbsp;when Townsend needlessly went into an aerial challenge with Speroni with a high leg.
There was no doubt about it, it was a straight red and Townsend had to walk to the dressing room past a hostile Holmesdale End. He didn't help his cause though when he made a '2-0' sign to the crowd.
His dismissal evened things up and Palace were able to come back into the game more. Burley changed things up with Jonathan Obika and Alex Marrow brought on for Alan Lee and Andy Dorman to give his side fresh attacking impetus.
The Eagles finally found their way through three minutes into the five minutes of&nbsp;stoppage time when Danns got to Nathaniel Clyne's cross before Marton Fulop, but Ipswich held out for the win.
Att: 15, 781 (Away: 2,020)
There were two changes from the side that took to the field at Oakwell with Neil Danns and Julian Bennett brought in to replace Darren Ambrose and David Wright.
On the subs bench there were places for three of the club's latest additions - Stuart O'Keefe, Jon Obika and Alex Marrow.
Crystal Palace: 1. Julian Speroni; 2. Nathaniel Clyne; 4. Claude Davis; 5. Paddy McCarthy; 8. Neil Danns; 9. Alan Lee&nbsp;(30. Alex Marrow, 84); 10. Owen Garvan; 11. Andy Dorman (29. Jon Obika, 75); 15. Kieron Cadogan&nbsp;(6. Adam Barrett, 51), 16. Wilfried Zaha; 19. Julian Bennett
Unused Subs:&nbsp;12. Calvin Andrews; 13. Lewis Price; 21. Kieran Djilali; 28. Stuart O'Keefe
Ipswich Town: 21. Marton Fulop; 2. Jaime Peters (23. Tom Eastman 90;) 4. Gareth McAuley; 6. Grant Leadbitter; 7. Carlos Edwards; 14. Jon Stead (16. Tamas Priskin, 88); 17. Andros Townsend; 18. David Norris; 19. Luke Hyam (3. Shane O'Connor, 90); 20. Tommy Smith; 48. Darren O'Dea
Unused Subs: 1. Brian Murphy; 22. Troy Brown;&nbsp; 24. Colin Healy; 34. Conor Hourihane