This essay develops and defends a diagnosis of Carroll’s (Mind 4(14):278–280, 1895) regress of the premises according to which the moral of the regress is that arguments are constitutively presuppositional. It is argued that this diagnosis allows to vindicate the key insights of the rule-following account of the regress, while overcoming the main difficulties that the rule-following account faces.