Cliff Owen/AP Photo

"I now have the great privilege of caring for people I went to war with many years ago, and people I have sent to war, and people who raised me in the profession when I was a youngster," the secretary said under the stares of the Senate Veterans Affairs committee. "For Any allegation, any adverse incident like this, makes me as - makes me mad as hell."

Forty patients reportedly died at a veterans' hospital in Phoenix while waiting for doctor appointments, but VA records somehow showed everyone was seen within two weeks. Whistle blowers now say the actual waits were months long, and allege employees were "cooking the books" to meet federal standards. In Fort Collins, Colo., reports suggest a clinic faked similar records. Meanwhile, the department continues to deal with a flood of backlogged requests, a symptom of multiple presidential administrations, although recent progress could clear the jam by 2015.

Shinseki, now facing calls by some in Congress to resign, and VA Undersecretary for Health Robert Petzel were hammered by Democrat and Republican alike for nearly two hours over the headlines. Even Sen. John McCain, R-Arizona, who is not a member of the committee, attended anyway to call it a "crisis of confidence" in the agency.

Here are some of the toughest exchanges they faced.

Shinseki: "I'm not aware in other than a number of isolated cases where there is evidence of that. But the fact that there is evidence in a couple of cases behooves us to go and take a thorough look. And that's why we have structured this audit so that a set of clinicians are not going to inspect their own areas."

Petzel: "We have worked very hard, Senator Isakson, to root out these inappropriate uses of the scheduling system and these abuses. We have been working continuously to try and identify where those sites are and what we need to do to prevent that from happening [...] the individuals are, as you mentioned, held accountable."

Isakson: "Would they lose their job?"

Petzel: "I don't know whether that's the appropriate level of punishment or not."

Shinseki: "It should be, yes."

Blumenthal: Shouldn't the FBI get involved?

Shinseki: "I will work with the IG to make that available to him if that's his response."

Shinseki: "I'm sure someplace in a large organization you're always going to have something like that. But this is part of the reason why I engage the veterans service organizations on a near monthly basis. If there are any straight shooters here it'll be them in terms of being direct with the secretary. This is why I've spent a good bit of time traveling the country, going to our facilities and talking to them about what's important and engaging veterans in those locations as well."

Shinseki: "And I can tell you, over the past five years we've done a lot to make things better. We're not done yet. And I intend to continue this mission until I've set aside either that goal or I'm told by the commander in chief that my time has been served."

Secretary Shinseki, a retired Army four-star general, suggested he could replace members of the department's upper management team depending on the results of an internal audit report due in roughly three weeks. Today's hearing was followed by hours more of expert testimony from independent veteran and health groups, as well as the VA Inspector General, whose own report is due August.

The Inspector General, Richard Griffin, stressed restraint in the investigation and cautioned that although reports suggest 40 died waiting for appointments in Phoenix his office had only reviewed 17 names thus far, further noting no positive link had been established yet.

"It's one thing to be on the waiting list and it's another thing to conclude that as a result of being on the waiting list - that's the cause of death," Griffin said.

"We are not going to rush to judgement at the sacrifice of quality," he continued. "We're going to nail this thing."

This report was updated at 4:50 PM.