Authorities are continuing their search for a 4-year-old Jonesboro boy whose father, who allegedly abducted him, was arrested Friday in what officials call a "third-world"-like New Mexico compound.

Abdul-Ghani Wahhaj, who suffers from numerous medical conditions, was allegedly abducted by Siraj Ibn Wahhaj last December. Siraj Ibn Wahhaj was arrested Friday with another Atlanta man, Lucas Morten, in Amalia, NM.

Taos County law enforcement officials found 11 children and three women, believed to be the children's mothers, also living there, but did not find Abdul-Ghani Wahhaj.

Officials believed little the boy had been there several weeks before, and are continuing to search for him.

A spokesman for the Taos County sheriff's department said the department suspected the boy was at the compound but didn't have enough evidence to go onto the property.

“Without some concrete evidence that this child is in fact there, we choose to do a surveillance from far, from a distant,” the spokesman said. "We did not see a child there in all of the hours of surveillance that we did do. We just didn’t see him.

“It’s not like we can just go kick someone’s door in and say we think this is the case. So, we took a cautious approach, so that we didn’t provoke anything that could lead to anyone being harmed, whether it be us, the people on the compound or the children.”

WATCH | What's being done now to find little Abdul-Ghani Wahhaj

That all changed last week, according to Sheriff Jerry Hogrefe, who received a tip from a Georgia investigator the children's welfare was in danger.

MORE | Suspected Clayton County child abductor arrested at New Mexico compound

On Thursday, Hogrefe issued a search warrant for the compound, described as an earthen berm in a local subdivision was allegedly occupied by Morten and Siraj Wahhaj.

The warrant came after a two-month investigation in conjunction with Clayton County and FBI investigators. Warrants for Siraj Wahhaj's arrest claim he took his son to perform an exorcism on the child and was denying Abdul-Ghani Wahhaj's medications.

Hogrefe issued the warrants after he received a message from a Georgia detective that came from inside the compound through a third party that said, “We are starving and need food and water.”

"I absolutely knew we couldn’t wait on another agency to step up and we had to go check this out as soon as possible," Hogrefe said, who added his agency believed the occupants were likely heavily armed and of an extremist Muslim belief.

WATCH | New Mexico official talks about why they decided to raid the compound

Both Morten and Siraj Wahhaj were heavily armed, officials said but were taken into custody without incident.

More ammo was found in the compound, which consisted of a small travel trailer buried in the ground covered by plastic with no water, plumbing, or electricity.

“The only foods I saw were a few potatoes and a box of rice in the filthy trailer,” said Hogrefe. “But what was most surprising, and heartbreaking was when the team located a total of five adults and 11 children that looked like third-world country refugees with no food or fresh water, no shoes, and basically dirty rags for clothing.”

Morten was charged with harboring a fugitive and Wahhaj was booked on his no-bond Georgia warrant for child abduction. The women were eventually released.

READ | New details in search for missing boy who may be without his medications

The mother of Abdul-Ghani Wahhaj called Clayton County police on Dec. 10, 2017. She told authorities that she hadn't seen her son since Dec. 1 when her father, Siraj Ibn Wahhaj, said he was taking him to the park.

The mother told police her son has to take medication because he suffers from seizures, developmental and cognitive delays and is unable to walk due to suffering a Hypoxic Ischemic Encephalopathy (HIE) at birth.

AND | 3-year-old missing without medication believed to be heading to New Mexico

A Clayton County judge issued a "Pick Up Order" for the toddler and he was placed on the Georgia Crime Information Center and National Crime Information Center Databases as a missing person.