Throngs of people pressed along barriers near the landmark Cibeles Square, waving flags and shouting "Long live the Pope" as Leo arrived in his white popemobile for the event.
Some tossed flower petals as he arrived in the square.
Some 1.2 million people were in the square and its surrounding streets, the Vatican and local organisers said.
Leo urged the crowds to live their Catholic faith by helping others in his sermon for the mass.
He said God "identifies with the poor, the downtrodden, those who are alone and forsaken".
The Pope also expressed hope earlier that Madrid would "remain a welcoming and inclusive city, where social life is inspired by genuine human values" in a message in a guestbook as he was handed the key to the city by its mayor.
Leo began his trip on Saturday with meetings with migrants and the homeless and a vigil with about 600,000 young people in Madrid.
His visit also includes stops in Barcelona and the Canary Islands, where he will meet migrants who have risked their lives crossing from West Africa.
He said he hoped the visit, his first to an EU country outside Italy, would set an example to the world for respecting "every human being" and urged leaders to stop dividing electorates.
"I am delighted that he is praying for us migrants and for our safety," Andrea Margarita, a 72-year-old Peruvian who arrived in Spain six months ago, said as she waited in the crowd in a wheelchair with her daughter.
After mass, Leo was scheduled to hold a private meeting with fellow members of his Augustinian religious order before meeting figures from the world of entertainment, sport and culture at a concert venue in central Madrid.
