If I would to do it one by one, it will take a lot time.
Then I found this wonderful plugin called Term Management Tools.
For those of you who find that you have too many categories and tags and would like to merge them together respectively, then use this tool.
It can combine terms, set term parents in bulk, and also swap term taxonomies with just a few clicks.
Here is how to use this plugin:
1. Get Term Management Tools at:
http://wordpress.org/plugins/term-management-tools/
2. After installing and activating the plugin, go to Admin page.
3. Go to Posts and click on Categories.
4. You will find the Bulk Actions drop-down.
5. If you want to combine a sub-category to the parent category, click Merge.
6. Then type in the parent or the main category in the box which you want the sub-category to combine together.
7. Check on the box next to your sub-category which you like to be merged. If you have many sub-categories, you can do them all together. Check all the boxes at the same time.
8. Click Apply button.
9. Now there the sub-category or categories are gone. They all named under the parent or main category.
10. You can see the number of posts increases as indicated on the right hand side.
Note: There will always be one category which CANNOT be deleted. It is a default category in WordPress.