Embattled Attorney General Eric Holder told the Senate Judiciary Committee on Tuesday that he has "stuck by my guns" during his time in office.