You say that the current sleeping arrangement “bothers” you. Any behavior that your son does that “bothers” you needs to be addressed and substituted with an alternate, more socially acceptable behavior. After all, it is just a behavior. By using some of the same techniques that you have found effective with him, gradually teach him that there are benefits to him sleeping in his own bed.
First, identify why he comes to your bed. Is he scared or does he feel lonesome? Prepare a plan that will help him feel more comfortable and secure in his own bed. Follow that plan every single night. It may include reading a bedtime story or saying prayers, but it needs to be done in his bed – not yours. Make sure there is a night light in his room. Turn on relaxing music. Prepare a chart that records how many nights he goes to sleep in his own bed, and perhaps another that records the number of nights he remains in his bed all night.
Together, set a goal and reward him when he attains this goal. You are forming and shaping new behaviors and habits. Be patient. This takes time. It took 10 years to learn the old habit. New habits cannot be established in just a few days.
We parents do what we feel is best for our children. Others can be so judgmental. It is doubtful that there will be long-term effects of this behavior. But it is our goal as parents to develop independence in our children as they get older. Your new plan will do just that.