Hmmm... you may want to get that checked out and sorted. Once the queen decides to live there you can have a big problem.
I had somethng like that a few years back with a bumble bee. All the time I had one indoors while everything was closed. I kept putting it outside, only to have it back inside within 5 minutes. So I checked it out. It was flying near a wall, then I noticed there was a piece of cement missing in between 2 bricks. And it went in there. To end up in my living room again, so clearly there is some opening under the window sill indoors somewhere.
In any case, I phoned council and they sent someone who filled it in. Thank goodness in time, no queen in between the outer walls yet. The bumble bee flew around the wall for a few days, clearly confused he couldn't get in anymore, hihi.
But you wouldn't want a nest, even though bees and bumble bees are basically friendly.