Does taking online Coursera classes count towards educational expenses? Been taking certificate programs, but not sure if it counts as an expanse or not

It could.

Ask the school if they are an "Eligible Education Institution". 

According to the IRS:

“Eligible educational institution. An eligible educational institution is generally any accredited public, nonprofit, or proprietary (privately owned profit-making) college, university, vocational school, or other postsecondary educational institution. Also, the institution must be eligible to participate in a student aid program administered by the U.S. Department of Education. Virtually all accredited postsecondary institutions meet this definition. An eligible educational institution also includes certain educational institutions located outside the United States that are eligible to participate in a student aid program administered by the U.S. Department of Education. “