Human rights groups have accused Saudi Arabia of detaining five relatives of a US citizen who pursued a commercial lawsuit in Pennsylvania against royals including Crown Prince Mohammed bin Salman.

The five detainees appeared briefly on Monday at the Specialised Criminal Court in Riyadh, which was established in 2008 to try terrorism cases, said Abdullah Alaoudh, Saudi director for the Washington-based Freedom Initiative.

A judge read out the names of all five but did not disclose whether they had been charged with anything or set a date for a new hearing, Alaoudh said.

The Freedom Initiative has denounced the proceedings, saying that "turning a private, commercial dispute into a basis for unjust detention" amounted to "a gross abuse of authority" by the Saudi judiciary.

Saudi officials have not responded to questions about the case.

The five detainees are relatives of 15-year-old US citizen Rakan Nader Aldossari, whose family filed a lawsuit in June 2020 on his behalf against former Crown Prince Mohammed bin Nayef.

The lawsuit by Rakan's father, Saudi businessman Nader Turki Aldossari, accuses bin Nayef and other Saudi entities of failing to honour a decades-old contract related to a refinery project on the Caribbean island of Saint Lucia.

The suit was later amended to include Prince Mohammed, stating that he had placed bin Nayef under house arrest and seized his assets, thus preventing bin Nayef from meeting his contractual obligations.

US courts have dismissed the suit, but Saudi authorities nevertheless detained Rakan's relatives earlier this year in apparent "retaliation", according to a statement released last week by the Freedom Initiative, Democracy for the Arab World Now and ALQST for Human Rights.

"The three organisations call on the Biden administration to demand the release of the Aldossari family members and end their persecution," the statement said.

The State Department declined to comment on the specifics of the case.

However in response to a letter from Rakan to US President Joe Biden, a State Department official said Washington would "encourage the Saudi government to be clear and transparent about the charges your relatives face and the grounds for those charges."

The response, seen by AFP this week, added: "We have raised concerns about suspected acts of transnational repression with the Saudi government repeatedly and will raise your case with our Saudi interlocutors as appropriate."