In basal adipocytes, glucose transporter 4 (GLUT4) is sequestered intracellularly by an insulin-reversible retention mechanism. Here, we analyze the roles of three GLUT4 trafficking motifs (FQQI, TELEY, and LL), providing molecular links between insulin signaling, cellular trafficking machinery, and the motifs in the specialized trafficking of GLUT4. Our results support a GLUT4 retention model that involves two linked intracellular cycles: one between endosomes and a retention compartment, and the other between endosomes and specialized GLUT4 transport vesicles. Targeting of GLUT4 to the former is dependent on the FQQI motif and its targeting to the latter is dependent on the TELEY motif. These two motifs act independently in retention, with the TELEY-dependent step being under the control of signaling downstream of the AS160 rab GTPase activating protein. Segregation of GLUT4 from endosomes, although positively correlated with the degree of basal retention, does not completely account for GLUT4 retention or insulin-responsiveness. Mutation of the LL motif slows return to basal intracellular retention after insulin withdrawal. Knockdown of clathrin adaptin protein complex-1 (AP-1) causes a delay in the return to intracellular retention after insulin withdrawal. The effects of mutating the LL motif and knockdown of AP-1 were not additive, establishing that AP-1 regulation of GLUT4 trafficking requires the LL motif.