Do you maintain consistent meals, ie eat the same kinds of things, or do you have lots of variety and rely on the calorie counting for accuracy?
Instead of calorie counting, you can have some basic home cooked meals, for which you do know the macros, then simply make small adjustments depending on what happens on the scales.
For example, I make 3 different lunches, each of which has similar calories and macros. This is constant. On top of this, I have my dinners and post workout meals. Depending on how the scales are moving, I will adjust my dinners or post workout meals accordingly. Eg less or more carbs at dinner. Or maybe more oil or butter with my lunch.
It sounds like you are making large reactionary changes, rather than keeping things very consistent with small adjustments. The same strategy can easily be applied to training as well. People are often too reactionary - eg adding in 4 extra cardio sessions or whatever.