It depends. You used the term that this was your second home which is a common tax term. In tax terms this means it is a home you use for personal purposes as a home when you like, but is not purchased for investment. The fact that you used it for a home office does not constitute business or investment use of the whole home.
If the sale of the home portion is a loss, it is not allowed to be used to offset gains from investment or business property sales gains. A home office could have depreciation for only the square feet of office space. The office square feet divided by the total square feet of the entire home would be the business use percentage. Keep in mind that it must be exclusive use and absolutely no personal use of the office space.
Please update here if you have more questions with as much detail as you can and we will help.
@fgur