If not already done, create a templates directory in your site root folder (see Configuration to change the templates directory name).
Then create one .html file per template in this folder. For example:
<div>
Here is my custom template!
This site base URL is: {{ site.base_url }}.
</div>
One your templates created, you can reference them from your source file using the template front matter parameter:
---
title: A page
template: my_template.html
---
## We're in our own template!
{{ content }}