There are plenty of articles out there telling potential homeowners that now is a great time to buy, because it is, but the decision to own isn’t that simple.

A new Zillow report puts hard numbers to the fact that while comparatively low monthly mortgage payments, coupled with increasingly expensive rents, continue to make homeownership a relative bargain to Millennials and other potential homebuyers, there is a catch.

The main issue comes when you factor in that not only do borrowers have to save for a down payment in order to get the best interest rate, but coming up with that money is getting increasingly difficult as rents and homes values continue to rise.

According to CoreLogic’s (CLGX) latest home price index, home prices nationwide, including distressed sales, increased by 6.4% in September 2015 compared with a year ago.

For starters, it is more affordable to buy than rent. Zillow compared a homebuyer making the national median income ($54,990) and purchasing the median-valued U.S. home ($182,500) during the third quarter could expect to pay around 15% of their income towards payments.

In reverse, if the same household chose to rent a typical home, it could expect to pay slightly more than 30% of its income towards median rent.

And this is even true for Millennials.

Trulia recently published a report adjusting its numbers specifically for Millennials. Usually, when Trulia crunches its home-buying numbers, it assumes a 30-year, fixed-rate mortgage with a 20% down payment for households moving every seven years.

But the problem with this model is that it doesn't fit the situations that average Millennials face. Instead, Trulia said that it is typical for young households (ages 25-34) to move every five years and only be able afford up to a 10% down payment.

Using this information instead, Trulia found that it is still 23% cheaper for that younger generation to buy a home than rent nationally.

But from here, Zillow began to break down what encompasses a down payment and explained how it’s just not that simple.

On one hand, rising rents and fairly flat income growth are making it hard to save for a down payment in the first place.

“In other words, renters’ money that could be going into the piggy bank for a future down payment is instead going into landlords’ pockets,” Zillow stated.

Then, there is the option of a smaller down payment. However, Zillow added that this option often comes with higher interest rates and private mortgage insurance (for down payments less than 15%), which both conspire to raise monthly payments and consume a larger share of monthly income.

It’s important to note that “needing to put 20% down” is a common misconception when it comes to down payments. There are homeownership programs that allow buyers to save on the down payment and retain savings for home maintenance and improvements. Today’s programs include grants, first mortgages with below-market interest rates and annual tax credits.

Even Freddie Mac’s CEO Donald Layton went on record asking for mortgage lenders to consider writing more low down payment mortgages in order for the government-sponsored enterprise to help increase access to credit to potential homeowners.

Meanwhile, on the other end of the spectrum, down payment amounts themselves are a rapidly moving target.

Here are some numbers that Zillow provided as an example:

As recently as 2012, just after the national housing market bottomed out after the recession, a 20% down payment represented a lot less cash. For example, a 20% down payment for a median Bay Area home currently costs roughly $50,000 more than it did in Q3 2012 ($153,600 for a median home valued at $768,000 currently, vs $103,360 for a median home valued at $515,100 in 2012). Homeowners in San Diego and Los Angeles would need more than $20,000 in additional upfront cash in 2015 than they would have in 2012 to purchase the median-valued home in their area.

And the issue isn’t exclusive to those markets. In Denver, one of the hottest markets in the country, buyers in 2015 needed an additional $18,620 compared to Q3 2012 in order to put 20% down on a median-valued Denver-area home.

Plus, to make matters even more complicated, Zillow added that there is a possible third wrench in the works due to the potential for rising mortgage interest rates.

Ultimately, while it is a great time to buy a home, rising rents and rising home values are making it very difficult for some potential buyers.