Sure, you can use old baskets to start seeds. It's a great idea You might want to line it with something to control water flowing out. Plastic liner would work well, but you would have to be careful of overwatering. If you don't mind a little drainage you can use something more porous like fabric, coffee filters or moss, or nothing even.

As for the tea bags (assume they are used), they would do better in a compost heap, worm box, or buried in the flower bed.