Arguments have always played a central role within logic and philosophy. But little attention has been paid to arguments as a distinctive kind of discourse, with its own semantics and pragmatics. The goal of this essay is to study the mechanisms by means of which we make arguments in discourse, starting from the semantics of argument connectives such as ‘therefore’. While some proposals have been made in the literature, they fail to account for the distinctive anaphoric behavior of ‘therefore’, as well as for uses of argument connectives in complex arguments, suppositional arguments, arguments with non-declarative conclusions, as well as arguments with parenthetical remarks. We argue that a comprehensive account of arguments requires imposing a distinctive tree-like structure on contexts. We show how to extend our account to accommodate modal subordination and different flavors of argument connectives.