Given A=[1−12−2], B=[2−1ab].
In general (A+B)2=A2+AB+BA+B2. For this to equal A2+B2, we need AB+BA=O.
Compute AB:
(AB)11=1(2)+2(−1)=2−2=0
(AB)12=1(a)+2(b)=a+2b
(AB)21=−1(2)+(−2)(−1)=−2+2=0
(AB)22=−1(a)+(−2)(b)=−a−2b
AB=[00a+2b−a−2b]
Compute BA:
(BA)11=2(1)+a(−1)=2−a
(BA)12=2(2)+a(−2)=4−2a
(BA)21=−1(1)+b(−1)=−1−b
(BA)22=−1(2)+b(−2)=−2−2b
BA=[2−a−1−b4−2a−2−2b]
Add and set to O:
AB+BA=[2−a−1−b−a+2b+4−a−4b−2]=[0000]
From (1,1): 2−a=0⇒a=2
From (2,1): −1−b=0⇒b=−1
Check (1,2): −a+2b+4=−2−2+4=0 ✓
Check (2,2): −a−4b−2=−2+4−2=0 ✓ …