Once I get to the warehouse and am doing deliveries, I know I don't have to record every house I stop at to deliver a package, but how exactly do I record it then?
If you're a W-2 employee, under the new tax law you can no longer any of your work-related mileage. It's well-explained in this web reference: https://www.buildium.com/blog/how-tax-reform-affects-your-2018-mileage-deduction/
(Work mileage is still deductible on Schedule C if you're self-employed.)
Unfortunately, commuting miles between your home and your regular work location are not deductible. (Since this is generally a known distance, you can multiply the round trip times the number of days you worked and enter the amount as commuting miles when you are working on your tax return and are asked for this information.)
Once you are ready to start making deliveries, record your beginning odometer reading (mileage) in a log book. When you finish for the day, record your ending odometer reading. Note the date and that it was for making deliveries for Amazon.
I use the old pen-and-paper method, but there are also apps that will track your mileage using the GPS in your phone.