All I could find about Cloudrest is this.

Cloudrest is an odd mixture of architectural styles, with buildings like strangler vines, built on top of older structures. The oldest of all ruins in the city, and in a few isolated spots throughout the Summerset, are made of coral, which must have been carried many miles away from the sea. The material and the style of the ruins strongly suggest that the Sload may have once counted Summerset as a part of their kingdom of Thras.

Building on what Valric has written. 

It's a city on a mountain, highest point actually on the Island, which for Altmer is culturally significant. It's also the city that enjoyed closest proximity to the Crystal Tower when it was up. In the third era it was ruled by a king. Especially poignant considering it has Sload features in its archetecture that it builds upon. 

My educated guess based on interpreting the scant lore and what I know of the Altmer is that the city serves as a gesture of the Altmer's domination over the island. Rather hard to build on top of a mountain, no? It's land-locked and rather impractical from a strategic standpoint, except that it is on a mountain and able to be defended, which further supports the idea of it being a gesture city, IMO. A representation of the conquering of the sloads and the other creatures that inabited Summerset. In Straag, my fanfiction, I set it as seat for nobility. 

That being said, you have to understand, Altmer society is organized into echelons. The Wise (teachers and priests) became the highest echelon, followed by Artists, Princes, Warriors, Landowners, Merchants, Workers, and enslaved beasts. If you want a humble beginning for your character, I wouldn't suggest that he's a member of the Wise, but rather more like a member of the worker or merchant echelon. 

Hope this helps.