The Eagles controlled the majority of the first-half, but a breakthrough appeared to have eluded them.
That was until the 44th minute when Guéhi picked out wing-back Daniel Muñoz at the back post, who opened the scoring via a smart finish with his head.
Ultimately it proved the difference between the sides. Reflecting on the performance with Palace TV, the captain said: "This is not an easy place to come to.
"I'm glad we all fought together, stuck together and got the three points. Luckily we had that character today, which is always good. It's something to definitely build on for the future. So I'm proud of everyone.
"It was one moment of quality from Dani [that decided the game] Especially with the pressure coming from behind him. It was a fantastic goal."
The skipper's excellent cross unlocked the Clarets defence, with Burnley having looked fairly resilient up until that point. But Guéhi was typically modest with his assessment of the assist.
"I just put it into an area. I don't think I can take any credit for that," he added.
"Honestly, more credit to Dani because he's always on the edge of those crosses and constantly making runs in behind.
"You know, credit to him for always being there. Like I said, it was a difficult chance. It's not easy to score that."