No, you're wrong. DEI says that you do not deny opportunities for people based on race, skin colour, sexual orientation, gender identity, able-bodiedness, or any other arbitrary criteria. You can't be merit based unless you offer opportunities to a diverse range of people, you offer them equitably, and you include everybody in your pool if candidates. If you're not doing that, you're not merit-based, because there may be some people with plenty of merit who are outside your pool.
Affirmative action says that you should provide extra benefits for those disadvantaged by circumstances. You provide the benefits so that those people can access the same opportunities as people who don't need them. Affirmative action levels the playing field.
In other words, people whose circumstances mean that they have less opportunity because of things that have happened in the past do not get the opportunities that more fortunate people get.
That's discrimination, inequity, exclusion. It doesn't matter if someone is the most qualified for your job, if they can't climb stairs because they're confined to a wheelchair they can't get to the job interview. Nobody should be treated differently, right? You shouldn't provide a ramp for people in wheelchairs, right?