A lot depends on what your needs and intentions are. 
   For any sort of target competition you have to fit it into the rules and practicalities of the specific game. 
   If you are going to hunt with it, it'll depend on the game and expected environment. 

Another factor is your willingness/ability to reload. Reloading opens a whole new level of potential accuracy and power,  but it also creates  new levels of complexity and expense.

If we knew a bit more we might be able to give you better information about possible options.

for casual shooting some of the less intense rimmed cartridges like the hornet would probably work fine. Though it has a reputation for a lack gilt-edge accuracy in some rifles, it seems to work well in others.  I have one on a BSA Cadet Martini that is both accurate and a joy to shoot---and so far all I have used is factory loads.

Another possibility is the 17 hmr round,   basically a 22 rf mg necked down to 17 cal.  It has proven to be an excellent little varmint control round.  I have it in one of the little savage "stevens favorite-alike" single shots.  As came from the factory it is a bit rough and has a very heavy trigger pull that only a liability lawyer would like.  with some better wood, a little work and a trigger job it'd be a great little rifle at a relatively low cost.  seems like they might have made 22 hornet version as well. 

you might look in the classified section on the site here and see if anything catches your interest.

go to:

(You need to Login or Register to view media files and links)

then click on the "single shot series"

I got my 17 hmr a few years back, I don;t know if they still offer it or not.  and mine needs some work to have an acceptable (to me at least) trigger pull.

you might also keep your eyes open on the classifieds for one of the BSA Cadets that is chambered for one of the more modest centerfire cartridges,  22 hornet, 32-20, maybe 38 special or something like that.

If you want to move up for some more bang in your rifle shooting, you might look at the CPA copies of the Stevens 44 1/2, the ultimate Stevens single shot.  A basic sporter configuration in the ubiquitous .30-30 could be shot outdoors with factory loads to your heart's content (or your check book's limits) and then you could do some very simple hand loading with the Lee hand loading tool that's been around for years and even make up some gallery loads to shoot it indoors.  Does that sound like what you were talking about?

Froggie

There are decent Stevens 44 guns out there in mild centerfire calibers that are still available. I have one in .32-20 that is a lot of fun. The nice thing about the CPA 44 i/2 repo's it that you can get additional barrels in various calibers later on. The Martini is also a fun gun. Lots of good options that are reasonably priced.

Converting a rimfire small Martini to centerfire is not a big deal once the mechanics of the system are understood.  32-20 is an easy caliber to build them in, either for fixed or breechseated bullets.  Same with .357 Mag. or .32 Miller Short.  I would recommend going with the thick sided versions, such as the 12/15.

David

I noticed you mentioned you're doing indoor informal target shooting. I think any of the cartridges mentioned would be either off limits, or dangerous for indoor shooting at most any range. 
If your interest is to buy a rifle caliber that's safe and legal for indoor ranges, then you need to look at centerfire pistol calibers like maybe  a .32-20 or .38 Special. Not typical "target" calibers in the competitive sense, but much better choices for a bit more bang indoors.

A low-wall or BSA Cadet in 357 mag would allow use of light 38 special loads for indoor target shooting and still have enough power with the right bullet for small game outdoors.  I'm not sure but think that a stevens 44 could handle it as well; but I'd check with someone much more knowledgeable for that

I think Browning chambers their new version of the lo-wall in that calibre.  I've seen several originals re-rifled and rechambered for it---even had one myself at one time.  I had Harry McGowan re rifle it.  It was a rework of a 32-20 with a really bad bore.  turned into a dandy little fun rifle.  even took a couple deer with 357s in it---close up shots in the brush using 158 gr jacketed flat points.