In Border select mode, click on the edge opening for the arm, and then switch to the front view and hold down SHIFT and pull outwards to start the sleeve.

I moved the edges down a bit and rotated them to an angle. Then I selected just the top sleeve edges and connected just them. In vert select mode, I selected the new edge vert and connected it to existing verts (make sure to switch to perspective and check the back to make sure to connect it properly there).

Then in Vertex select mode I moved the verticies around so that they fit the shape of the arm better. Once I had the shoulder started, I selected the border of the sleeve and while holding down SHIFT I pulled it furhter down.

Continue working your way down, and use scale on the selected edges when you need to make them smaller because this will make them smaller from other angles as well and reduce the amount of fixing you'll need to do from the top/side views.

It's hard to get a good second angle on the arm so you'll have to do a lot of tweaking in perspective.

Another important thing to take into account is how the arm will deform when rigged and animated. With this baggy simple sleeve, a lot of geometry really isn't needed, but if you have too little geometry in the joints, they will deform poorly. I decided that I needed one more loop at the elbow, so i selected the edges just below the elbow and clicked connect and adjusted the Slide so that the new edge was created near the elbow.

Time for some more optomizing!

When you're tight for polys, any extra or not-totally-nessecary edges are important to remove. The arm has a lot of edges going around it - more then it needs, so I went back in and started welding together any areas that could afford it.

Do a general all-over pass for anything you can optomize or tweak at this point, and set the smoothing groups on the sleeve.