Closet studs turned, wall is uneven

I am currently working on a kitchen remodel. When I opened up the wall, I discovered that most of the studs were uneven. The previous cabinets had been affixed to a 1x that ran the whole length of the wall, and it was shimmed to account for bow, then plastered over. The real concern was when I got to the part of the wall in the kitchen where there was a closet behind the wall. The studs on the closet are turned so the face is showing. It’s about .5” off from the rest of the wall. My question is, should I knock out the plaster and turn the studs, or can I just shim it out and drywall over it. My biggest concern is that the movement in the wall will result in lots of cracking.