Money worries us more than our health, particularly for the "negative equity" generation.
These are people in their 30s, 40s and early 50s who bought homes during the boom, but are now saddled with large mortgages and properties worth less than the loans secured on them.
The second most significant worry for most people is their family, with health coming in third place, a new survey shows.
However, health is the main worry for older people, according to the survey commissioned by insurance company Royal London and carried out by iReach on 1,000 people.
Getting promoted in a job is a major worry for those between the ages of 18 and 34, the survey says.
More than four out of 10 people aged between 35 and 54 said their financial situation was their main concern.
Joe Charles of Royal London said that generation was under huge financial pressure.
"Many in the 35 to 54 age group will have to contend with large mortgages, many of which are in negative equity, plus the cost of childcare, the prospect of working until they are considerably older than their parents' generation have or had to, and a still-challenging job market.
"All of this impacts in terms of euros and cents."
Mr Charles said the majority across all age groups have no life assurance, but this is highest for those in their 30s, 40s and early 50s.
Money worries also top the list for those aged between 18 and 34, with this group often referred to as Generation Y.
Some 13pc of this age group said job progression was their biggest cause of worry, which was nearly twice that of other age groups.
Many in the 18 to 34 age group have probably found it difficult to advance their career as those older than them and ahead of them on the employment ladder have been less likely to move jobs due to the economic uncertainty of the past few years.
High levels of youth unemployment, impacting many of their friends and peers, may have coloured their answers, Mr Charles said.
The Royal London survey also reveals that health becomes the biggest worry for those aged over 55.
Mr Charles added: "While financial troubles topped the poll for both men and women across all ages, the second most common cause of worry revealed an interesting gender divide - it was health for men, but it's family for women.
"However, the numbers worrying about relationships are relatively low for both men and women in all age groups, so it's good to see that for both sexes at least, partners aren't the biggest source of anxiety in people's lives."