This study aims to determine the optimal design and configuration of Flexible AC Transmission System controllers, optimizing parameters such as power flow and congestion management in the competitive electricity market. The technique proposed in the paper generates multiple solution sets in the power system according to the location and control parameters of the Static Synchronous Series Compensator, Static Synchronous Compensator, and Unified Power Flow Controller. The minimization of transmission congestion, real power loss, voltage deviation, and controller installation cost in the proposed optimization technique considers the power flow equations, bus voltage magnitude limit, line thermal limit, generator limit, and controller operation limit. The optimization technique is carried out using the Newton-Raphson power flow method to check solution feasibility and update the power system operating conditions. The suggested Hybrid Capuchin Fruit Fly Optimization Algorithm enhances the transmission capability, congestion management, voltage stability, operating cost, and secure and reliable operation of competitive power systems operating under varying loads. In the IEEE 14-bus power system, for example, a proposed congestion management strategy for the system is capable of reducing the power flow on the most congested line (Line 15 (Bus 7-9)) from 28.8804 to 24.9709 MVA, which is a 13.54% reduction. Furthermore, the Unified Power Flow Controller has the lowest cost of congestion of $487.84/hr, just 3.85% and 0.86% lower than the Static Synchronous Series Compensator and Static Synchronous Compensator, respectively. The results show the effectiveness of the proposed control optimization-enabled controller coordination framework in enhancing congestion management and economic performance.