This is a question that is often asked, but I find a lot of what is written on the topic unhelpful.
The argument often quoted is: “If the heathen are condemned to hell because they haven’t turned to Jesus, and they have had no chance to hear, isn’t this unfair?”
It is true that “Salvation is found in no one else, for there is no other name under heaven given to mankind by which we must be saved.” (Acts 4:12).
There is no other way anyone can be saved, whether Jew or Gentile.
This is also true whether they live before or after Jesus died on the cross.
As Romans 3:25 indicates, God “had left the sins committed beforehand unpunished”. These are the sins committed by Jews such as Abraham and Moses. They were left unpunished because God knew that he would be providing the atonement for them in the future. The sacrificial system was only something to point towards the work that Jesus would do in the future, it was unable to clear sins (Hebrews 7:11, 18-19; 9:9-10; 10:1-4).
People who lived before Jesus died were saved because they put their faith in God, like Abraham did (Romans 4:3). They too had the good news proclaimed to them (Hebrews 4:2,6), they didn’t know the mechanics of how their sins would be dealt with, but they were to trust God, throwing themselves on his mercy, like the Tax Collector did in the parable, who also was pictured before Jesus died (Luke 18:13-14).
This discussion so far has been about Jews who hadn’t heard about Jesus, how about Gentiles? Well, Gentiles do actually have knowledge of God. Romans 2:13-15 says:
 For it is not those who hear the law who are righteous in God’s sight, but it is those who obey the law who will be declared righteous.  (Indeed, when Gentiles, who do not have the law, do by nature things required by the law, they are a law for themselves, even though they do not have the law.  They show that the requirements of the law are written on their hearts, their consciences also bearing witness, and their thoughts sometimes accusing them and at other times even defending them.)
Paul also says in Romans 1:18-20:
 The wrath of God is being revealed from heaven against all the godlessness and wickedness of people, who suppress the truth by their wickedness,  since what may be known about God is plain to them, because God has made it plain to them.  For since the creation of the world God’s invisible qualities – his eternal power and divine nature – have been clearly seen, being understood from what has been made, so that people are without excuse.
So they have sufficient knowledge of God and what he requires. If they do what their consciences tell them to do, then they will be saved. However, as we all know, we cannot totally obey our consciences perfectly, so we are all sinners in God’s sight. “There is no difference between Jew and Gentile, for all have sinned and fall short of the glory of God” (Romans 3:22b-23a).
Now I said above that God expected Jews to trust God, throwing themselves on his mercy. I presume that the standard that God expects of Gentiles who haven’t heard of Jesus is the same as what he expects of Jews in the same situation. If they acknowledge that they are sinful and unworthy before God, and throw themselves on his mercy, rather than relying on their own insufficient works, then they will be saved.
The question is not whether the heathen are condemned because they haven’t heard of Jesus, it is rather, what have they done with the information they have received about God? If they do not turn to him in faith, and instead are “suppressing the truth by their wickedness,” (Romans 1:18) then they are rightly condemned, just like those who have heard of Jesus.
Ultimately we do not know whether an individual “heathen” has turned their life over to God or not. It may be many or it may be few who have done this. Certainly in the Old Testament we hear of non Jews who decide to join the Jews or otherwise show that they are believers. Women in Jesus’ genealogical tree such as Rahab and Ruth are examples.
Other examples of non Jews who worshiped Yahweh are:
- Melchizedek, King of Salem, who Abraham gave a tithe to (Genesis 14:17-20),
- Abimelek, given his interaction with God seems to be a worshiper (Genesis 20),
- Jethro, Moses father in law, who is described as priest of Midian (Exodus 18:1), and
- Balaam who lived in Pethor on the Euphrates (Numbers 22:1-24:25)
This would imply that people in Salem, Pethor and Midian all worshiped Yahweh, though in an imperfect manner.
Where does that leave us as far as evangelism is concerned? Well, the heathen are just like everyone else, we need to tell them about Jesus, so that they know the mechanism by which God will save them, we need to tell them that if they do not turn to God they will be condemned. Some may have turned to God, but we do not know who or how many, we just need to get out there and tell the Good News to as many as we can, so that by all possible means we might save some (1 Corinthians 9:21-23).