Yes, you can add images as well as videos to the product gallery. To add images to the gallery go to the Edit product page and add as many images as you want in the product gallery. 

For adding videos open the gallery images in the WordPress media library. Select the image in that you want to add the video. You will see an input field named Video Link at the bottom of the Attachment Details. Just paste your video link there (YouTube, Vimeo, Dailymotion, Facebook, and Self-Hosted), and the video will be set with that image.