I want to start a new project, have devbox manage the environment and have direnv load that environment when I cd into that directory.

Make the new directory, cd into it and then:

devbox init
devbox generate direnv
devbox add <dependecy>