David Gareja is an old monastery complex with caves carved into the rocks a few millennia ago. It’s right on the border with Azerbaijan and you can do a day trip from Tbilisi.