Using AMD syntax, you can pre-load modules either inline on the page or via script tags (if the AMD module is named). You can use this technique to speed up your page.
Another technique shown in this page is using an AMD "wrapper" to modularize non-AMD code. For libraries such as underscore or backbone which are not designed for AMD, you can include the un altered version of the library for 1 additional HTTP request.
Combining the two techniques above or using an AMD optimizer can further improve performance and remove this penalty.