Not necessarily. I would inspect the larger tree to look for defects or signs that it is in a long slow decline or self retrenchment. If those other trees are the same species as the larger tree they could be root grafted and potentially bringing valuable resources to the larger older tree. It’s like if you were living with your grandma. To some you would look like a leech but to her when you take out the trash, mow the lawn, and keep an eye on things you are a godsend.
Just keep an eye out for structural issues with the smaller trees interfering with limbs on the larger tree and let them coexist. Long term those smaller trees might be the replacements.