In a small bowl, make a seasoning mix and combine together Cajun seasoning, pepper, garlic powder, onion powder, and sugar. Set aside.
Whisk 1 Tablespoon of the prepared seasoning mix into the buttermilk.
Pour the buttermilk mixture over the chicken. Cover with plastic wrap and place in the refrigerator. Let the chicken marinade at least one hour to overnight.
When you are ready to fry the chicken, in a medium whisk together the remaining seasoning mix with the flour and cornstarch. Place 3-4 Tablespoons of the buttermilk marinade and stir it into the flour mix. You'll want to make chunky pieces of flour to give the chicken some good crispy bits.
Dredge the chicken in the flour mix and dip back into the buttermilk and dredge in the flour a second time. Press any of the chunky flour pieces into the chicken.
In a large heavy pot or deep frying pan add 4 cups of oil or enough to cover the chicken ¾ of the way. Heat the oil over medium high heat until a little flour dropped in the pan sizzles immediately.
Add 2 pieces of chicken to the hot oil and fry 5-6 minutes, flip and repeat with the other side.
Once cooked through, Remove the cooked chicken to a cooling rack and repeat with frying process with the remaining chicken.
To assemble the sandwich, spread mayonnaise on the brioche buns, add the chicken breasts, top with pickles, and top bun.