There are two exemptions here: #1 your exemption from counting days for the substantial presence tests, which has allowed you to file as a nonresident alien for 2016 and 2017. #2 your exemption from tax on your fellowship income, due to the tax treaty with France, article 20, for two years from the date of arrival.
Assuming you arrived in 2016, on 1 January 2018 you start counting days for purposes of the SP test. If you count 183 days or more in 2018, you will be taxed as a resident and must file form 1040. See IRS publication 519, section on "Substantial Presence Test"
Assuming you are still a resident as of 31 December 2018, yes, you and your husband can elect to file a joint return. See the same publication, section on "Nonresident Spouse Treated as Resident".
Note that the exemption for your fellowship income ends on the day before your arrival date two years before. For example, if you arrived 18 August 2016, fellowship income received through 17 August 2018 is exempt from federal income tax, because this is a benefit that can be taken by a US tax resident who is not a US citizen or green card holder (ref. US-France tax treaty, article 29, paragraph 3(b)). See same publication, section titled "Students, Apprentices, Trainees, Teachers, Professors, and Researchers Who Became Resident Aliens" for how to report.
Be aware that since you will be taxed as residents, you have to report your worldwide income from all sources on your US tax return, including any income from investments outside the US. You also may have to report any financial accounts held outside the US.
