Gabriel's own goal in the first half was enough to earn Southampton their first ever win over Arsenal in the FA Cup.
The result brings an end to Arsenal's six match unbeaten in all competitions.
Before the match, Arsenal boss Mikel Arteta said captain Pierre-Emerick Aubameyang had traveled but was unavailable to play due to "personal reasons."
Southampton started brightly and James Ward-Prowse came close to scoring inside five minutes but his corner rattled the crossbar.
The hosts tooks a deserved lead after 24 minutes when Gabriel deflected in a low cross from Kyle Walker-Peters.
Arsenal improved in the second half and Rob Holding had a good chance to equalise from a corner but failed to hit the target.
Nketiah failed to score with a decent chance in stoppage time and Southampton held on. They will face Wolves in the fifth round of the FA Cup in February.
"We had in the first half more punch and more power," Southampton manager Ralph Hasenhuttl told BT Sport. "In the second half it's normal they have a lift. We must make the second goal.
"We made one or two chances -- Shane Long should have got a penalty. We had early ball wins, but our last decision-making was not always the best. In general our work without the ball was good. It was a good team performance today."
It was the third in the last four seasons where Arsenal have failed to progress into the fifth round of the competition.
"Really sad to be out of the competition," Arteta told BT Sport after the match. "Disappointed to be out and the way we conceded. We had issues in the first half and unforced errors as a result of the high-press they do.
"In the second half we had two or three big chances but not enough. It was an even game. In the first half they won a lot of second balls. In the second half we added more quality.
"We became more dominant and created chances but it wasn't enough. "We are looking in the market and see what we can do."