Potassium currents generated by voltage-gated potassium (Kv) channels comprising α-subunits from the Kv1, 2, and 3 subfamilies facilitate high-frequency firing of mammalian neurons. Within these subfamilies, only three α-subunits (Kv1.4, Kv3.3, and Kv3.4) generate currents that decay rapidly in the open state because an N-terminal ball domain blocks the channel pore after activation—a process termed N-type inactivation. Despite its importance to shaping cellular excitability, little is known of the processes regulating surface expression of N-type α-subunits, versus their slowly inactivating (delayed rectifier) counterparts. Here we found that currents generated by homomeric Kv1.4, Kv3.3, and Kv3.4 channels are all strongly suppressed by the single transmembrane domain ancillary (β) subunits KCNE1 and KCNE2. A combination of electrophysiological, biochemical, and immunofluorescence analyses revealed this suppression is due to KCNE1 and KCNE2 retaining Kv1.4 and Kv3.4 intracellularly, early in the secretory pathway. The retention is specific, requires α-β coassembly, and does not involve the dynamin-dependent endocytosis pathway. However, the small fraction of Kv3.4 that escapes KCNE-dependent retention is regulated by dynamin-dependent endocytosis. The findings illustrate two contrasting mechanisms controlling surface expression of N-type Kv α-subunits and therefore, potentially, cellular excitability and refractory periods.