Not really that unusual at all. Brake pads normally do not retract fully merely upon release of the pedal/lever. Rotors have a small amount of runout, even within spec. If the sound stops when you gently drag the brake, you have probably isolated the cause. Almost all bike w/disk brakes have some slight drag.
Another test is to run the bike up to speed, then slow down to a crawl (the slow, quiet speed you mentioned) using only the front brakes. If the sound is absent, you have another confirmation.