I want to play video in my site with Authentication, so that only registered users can view it.
I mean:
Make a custom video player with some sophisticated algorithm for changing the URL of the video stream and encrypting the stream data so only your player can view it.
There's no real way to protect HTML5 videos. The browser usually lets you right click -> download it. The way YouTube protected against that is an overlay that blocks right clicks... but its nowhere near perfect.
There has been some discussion about DRM at W3 bug tracker, but there's no real solution (yet). One intesting approach that was mentioned there can be found here. Also see the main post, which provides some interesting insights on that.
You can use Flash Media Server and Amazon Cloudfront for secure streaming.
Note: Amazon cloudfront is paid service.
Adobes RTMPE protocol was made for pretty much this purpose. Adobe Flash Media Server product will transcode your videos on the fly, delivering it to your clients in a protected manner. Essentially, it is a DRM solution. Your clients will use Adobe Flash Player or Adobe AIR to view the videos, and in a perfect world no unauthorized user will be able to view the content.
Whether DRM or RTMPE works to the industrys or users satisfaction is another matter, of course. As is the case with whether the system can be broken for sake of pirating the content.
Although you can block the interface to download the video in HTML5 and Flash, this is not a guarantee it won't be pirated. Just look at the sheer number of YouTube downloaders out there and you'll see what I mean.
HTML5 isn't the ideal thing for DRM, nonetheless. Flash provides DRM trough RTMPE / RTMPS, but tools to break the encryption are widely available. Microsoft Silverlight is a third technology that provides DRM, but it won't work on Linux-based computers. I've never looked into Silverlight DRM, but I'm pretty sure it can be cracked as well.
All the above-mentioned technologies support authentication; before serving a video to a client, you'll simply have to check the user is logged in. This is the easiest with HTML5 if the video is served from the same server, but it can be verified for all other technologies as well by using a unique URL to serve the video to the user and then have the video streaming server contact the server where logins are handled to check if a user / IP address is authorized to view the video stream.
To summarize: if you value cross-platform compatibility, choose HTML5 and block the right-click menu to prevent downloading; otherwise, you can use either Flash or Silverlight DRM - but be aware that neither of these is perfect either.