The Rambam writes in the Morah Nevuchim that providence (hashgacha) is according the to intellect. That means that it comes via the intellect. He also writes that hashgacha is only when G-d aides people or the world. Nothing evil is called hashgacha. A prophet may die when a house falls in on him, but that is not hashgacha but is rather the lack of hashgacha. The prophet must have been distracted for a moment and have disengaged his thoughts from G-d and therefore been susceptible to the evils inherent in matter.
Thus, evil is an absence of hashgacha or in modern terminology and withdrawing of G-d from the world. But more precisely, for the Rambam, it would be man withdrawing from G-d, deactivating his intellectual connection with G-d. That allows nature to take its course. That course can be either good or bad. It can either bring life or death, but it is not direct hashgacha. Hashgacha is only positive and only occurs in those people who are actively contemplating G-d. This theory then explains what the Rambam writes regarding prophecy. He writes in the Mishne Torah that good prophecies will always come true but bad prophecies may not come to pass. For example, in the book of Yonah we see that Nineveh is not destroyed. This fits very nicely to the aforementioned theory. A good prophecy represents G-d's hashgacha so it will happen. But a bad prophecy only means that G-d will let nature take its course. Nothing evil comes from G-d (Talmud). When G-d says an evil will occur to a person or to people it can only mean that He is predicting that evil to occur, not that He will cause it. G-d knows the nature of man and of natural phenomena. When he predicts an evil He is only saying that based on man's normal behavior, this is what will happen.
This answers the Raavad's question to the Rambam in Hilchot Teshuva. There the Rambam writes that the prophecy of 400 years being exiled in Egypt did not take away the Egyptians' free will. Each Egyptian could have not afflicted the Jews. The Raavad asks, but what if all of them had chosen not to participate, then the Jews would not have been enslaved?! And the prophecy would not happen!
The Rambam answers simply, yes. No negative prophecy is guaranteed to occur. G-d is merely predicting the way of nature. Likely there will be a famine. Egypt as the "House of Bread" will likely be where the Jews seek out food. They will probably end up staying there. The locals will probably start to suspect them and then enslave them. They will resist and run away, leaving Egypt once more. Negative occurrences are never the result of G-d's direct action and therefore they can never be sure things. In fact this is what the Raavad writes. He compares G-d to a star gazer who knows the future simply as one who looks at nature and makes a prediction based on past occurrences.
As a side note, I would like to add that there may be another way to explain the Raavad. He may be saying that just as my knowing past events does not affect the free will of the people involved in those event. G-d for whom the future and past are no different since He is outside of time, knows the future in the same way that we know the past. Thus, His knowledge of the future does not affect the free will of those involved in it. This is what the Raavad means by comparing G-d to start gazer. A star gazer sees past events but does not affect them in any way. Before I was saying that the Raavad means that a star gazer can predict the future based on previous alignments of the stars, but has no perfect knowledge of the future. Now I am suggesting that all the Raavad means is that the start gazer sees the past events and that is equivalent to how G-d sees future events.