Stop looking at your hands. The neck does not move nor the frets and strings on. It's like not looking at your fingers on a keyboard, the keys do not move and when you trust your fingers to go where they should, your typing speed and accuracy goes up.
Start by trying for 5 minutes when you start each practice session, I believe in 2 weeks or less you will be moving the amount of time up because you want to. You have spent a lot of time building muscle memory, let it work for you now, start trusting your body will do what you want even if you do not directory tell it to.
Most of the time when I'm finger picking with my right hand, I do not even think about it, it just plays. It's kind of odd to look down at it and just watch it go. A new pattern does not work that way for a while.
Next time you watch Robin Trower's Bridge of Sighs video, watch his eyes, I bet there closed 90% of the time. Free your mind, give it more resources. It may be a lot like learning to sing, hard at first, but it will come quickly.
Next time you pick up your guitar, sit down or put your strap on, close your eyes. Next put both your hands on top of the guitar. Then pick up both your hands and move your right hand to where you think it would be ready to strum, grab a "A" bar chord at the 5th fret with your left. Don't think about it, just let your left hand go. Open your eyes, see were your hands are.