Because they will be at a different focal point than the one in the middle.
Watch something on the TV now and hold a magazine up at arms length so that it matches the size of the TV. That's what your 19" screens will look like out of your peripheral vision. You'll have to change focus between them and the main TV.
Also, you'll only have to shift your view slightly left or right to result in things overlapping in your vision.
This. If you're older or wear glasses, frequent focal accommodation this is especially difficult. Binocular vision further complicates things, and the combination will likely make it impossible for your brain to 'merge' the images. Simply maintaining your head in the same position is intolerably uncomfortable if you plan to spend significant time at your PC.
Aside from all this, Eyefinity will not work in this situation. 19" 4:3 LCDs, likely 1600x1200, cannot be mixed/matched with 32" 1360x768 sets.