Does it cost anything to use the CBS All Access channel on my Apple TV?

No. You can browse and watch clips for free. However, in order to view premium content on the CBS All Access channel, you'll need to subscribe to the service, which is $5.99 per month, plus any applicable tax for the Limited Commercials plan, and $9.99/month for the Commercial Free plan*.

*Live streams like your local CBS station still have commercials and, due to streaming rights, a few shows play with promotional interruptions.