To speak about colonialism as one historical event is maybe in fashion (mainly among the lefties) but it is far from truth. You have to differentiate among different kinds of colonisations. To start from the beginning, the Portuguese started their colonisation for commercial reasons, and remained so quite long until the Dutch overtook them. This kind of colonisation I would call the soft colonisators. The Spanish on the other hand from the very first moment exposed their cruel and arrogant intentions. As said first they come to plunder with sword and cross, but very soon understanding the richness they have in hand, they abandend the cross and stayed with plundering. The English and French started to colonise other countries as a by product of the war and competition between themselves and with the Spanish. Their intentions were not always criminal as in the case of the Spanish. The prove of British intentions as colonisators can be seen in America, where the British wanted to protect the American Indians, and created border between the settlers and the inland territories. This was one of the reasons why the American settlers started the war of independence ( not the tea tax as many may think).
A very different story was the Belgian story. Their colonisation adventure was all about plunder, rasism and deceit.
The very worst big scale colonial power was the Russian empire, and also the most prolonged one. It started with Ivan the terrible in 16 century and was partly dismantled 20 year ago. The Russian colonialism built on brute power, brought to all the lands they occupied all the ills of their own criminal regime.
Yet the very worst colonial power of all were the Muslim occupants, whose policy of enslavement of local non Muslim people was maybe even more cruel and devastating than that of the Belgians. They imposed not only their discrimination based rule on the occupied nations, but have done all in their power to annihilate the local culture and faith. As a result of it, you have almost no Christian population in any of the previously Christian lands of north Africa and very few leftovers in the Middle East, no Zarathustrians in Iran, no non-Muslim population in big parts of East Africa, no Budhists in India, no Hindu or Budhist population in Bangladesh and Pakistan, or even Indonesia etc. (As contrary to it in India thrive today 200 million Muslims among the non Muslims).
This islamization of huge teritories happened out of power off sword and not the power of submission or peaceful persuasion. Even today in many Muslim countries converting from Islam to other religion is a criminal offence punished by death.