I briefly described above that a Unix process has two different user IDs (or uids, for short): a real one, and an effective one. When you log in, both uids of the login shell are set to the uid value of your user account. When starting a setuid application, the real uid remains unchanged, while the effective uid is set to that of the user who owns the program file. A program can query the values of the real and effective uid by using the getuid and geteuid system calls, respectively.
Changing these uids is a tad more complicated (this is Unix, after all!). The first problem is that this doesn't work quite the same across all Unix platforms. BSD based Unix systems use a different model for dealing with user IDs than those based on System V or POSIX. Both have roughly the same set of functions for dealing with this, but their semantics are slightly different. Unfortunately, these slight differences can make a tremendous difference for the security of your program!
Normally, these function calls are restricted to the root user. Normal users are of course not permitted to change their effective or real user ID at will. However, in a setuid process, the rules are less clear-cut as you will see below. As a rule of thumb, all system calls discussed below allow non-root users to assign the real uid to the effective one and vice versa, but never to change to a different one. Phrased yet in another way, you can move your uids around, and lose some, but you never gain additional privilege.
Before we dive head-on into the snake pit, let's me make some definitions that I hope will help you not getting tangled up. First, what I've previously called setuid applications I will refer to as set-user applications for the rest of the section, because below we'll talk a lot about a system function called setuid, and talking about two different things that go by the same name is a great way to mess up everyone's mind.
Second, I'll use the terms user uid and privileged uid quite a lot. Think of the user uid as the uid of your account. The privileged uid is the one assigned to the set-user application. Initially, when the program starts, the real uid of the process will be the user uid, and the effective uid will be the privileged uid. More precisely, the user uid is a numerical value, while the real uid is a variable beloging to a process. The same holds for the privileged uid and the effective uid, respectively.
Ready? Here comes the snake pit.