Hmm... I understand the feeling about being profiled, then denied something that is expected to be open to all people... but this situation reminds me of a strange thing I observed several months ago.
I live in an area of Tokyo where it is not known to be popular to tourists (hooray). There is nothing tourist-y around the main station, nor is there anything tourist-y around my general living proximity. I do see tourists, but they are usually staying at an AirBnB or something in my area.
Several months ago, I was walking to my local grocer, where there is a bus stop situated right in front of it. As I was walking, I noticed a long line of, what appeared to be, Chinese men, filing out of the bus, led by an Asian woman (she was probably Chinese, too). I know I am profiling, too, but we all do it, so cut me some slack! Anyway, it wasn't the fact that they were Chinese that caught my attention. It was because they were walking all in single-file, and they appeared to be between mid-20's and 30's. All quite fit, tall. All men. None of them carried any bags, backpacks, luggage. It just seemed odd... usually, for a tour group, you'd see a mix of different people-types, and you could really tell that they are tourists. This situation was strange.
Anyway, to cut a long story short, nothing happened. They just walked off somewhere, following this woman, down the main street and off to a side street.
I didn't pay much attention to them at that moment, but, thinking about it later, I thought that they might be part of a military unit... they appeared like they might fit that profile, with their age and fitness level. Plus, the whole situation of them not carrying any bags, looking around curiously, following a woman in single-file. What Chinese tourist group follows someone in single-file?
Of course, I cannot confirm any of this. It is just a gut feeling. But if I had observed something suspicious like that, perhaps the Japanese SDF being cautious is warranted...