Well, at the most basic level of physics that's right (M1V1 = M2V2), except:
1. The gun weighs anywhere from six and a half to eight pounds, and the bullet weighs (Most likely) 55 or 62 grains, and a grain is just a hair over two one-thousandths of an ounce;
2. The impact footprint of the rifle is the buttplate, pistol grip, and held area of the forearm, in turn transmitting the recoil to an adaptive biological structure that itself weighs 15 to 20 times what the gun does, while the bullet's impact footprint is the size of a pencil point;
3. The total energy may be same, but the impulse is not. All of the bullet's energy is immediately available at impact, how long it takes to expend it at that moment depends entirely on the target, not the bullet. On the other hand, numerous aspects of the rifle (Progressive burning powder, friction of the bullet in the rifling, recoiling parts working against the mainspring and buffer, a muzzle brake using residual gas to lessen recoil after the bullet is gone) and the musculo-skeletal response of the shooter all work to spread that force out over a much longer relative time.
DUmmies, please stay away from science, you are the poster children for 'Don't try this at home.'