Ross Perot appears to be headed for a confrontation with the Senate over his refusal to testify this month at a hearing on American servicemen missing in Southeast Asia.

After a private meeting Thursday of the Senate Select Committee on P.O.W.'s-M.I.A.'s, a committee spokeswoman said today, "The committee is unanimous in feeling that we do have to have his testimony."

The largest and most influential organization of families of missing American servicemen urged the Senate to compel Mr. Perot "to appear in open session and provide evidence to support previous statements that American P.O.W.'s-M.I.A.'s are alive in Southeast Asia."

Mr. Perot earlier this week canceled his appearance, saying in a letter to Senator John F. Kerry, the Massachusetts Democrat who heads the committee, that he did not want to submit to the "political circus" that the hearings "will tend to become." Mr. Perot also said he could not appear at any alternative date before the Presidential elections. Alternatives Are Considered

"It is not feasible to wait until after the election," the committee spokeswoman said today. Neither Mr. Kerry nor the committee's vice chairman, Senator Robert C. Smith, Republican of New Hampshire, was available for comment today on the status of negotiations with Mr. Perot. But Senate aides said the committee had discussed issuing a subpoena to compel Mr. Perot to appear or, perhaps, taking a formal deposition from him.

The conflict over Mr. Perot's appearance is significant because the hearings were looked upon by many of his supporters and his opponents as an opportunity to examine Mr. Perot's pointed assertions on issues about P.O.W.'s and -M.I.A.'s, including his belief that hundreds of American servicemen were left behind in Southeast Asia at the end of the Vietnam War.

The hearing was also expected to be a forum to examine Mr. Perot's confrontation with the Reagan Administration over policies on P.O.W.'s and M.I.A.'s.

In recent months, new details about that confrontation have come to light, among them that Mr. Perot carried on a private dialogue with senior Vietnamese officials between 1988 and 1991 even as a senior Presidential envoy was carrying on formal negotiations with Hanoi.

Mr. Perot, through his spokesman, has also conceded that he may have solicited private investigative material to attack the reputation of a senior Pentagon official, Richard Armitage, who helped supervise its policy on the matter. Political Worries Are Strong

Mr. Perot's dispute with the Reagan White House over P.O.W.-M.I.A. policy led to his estrangement from Mr. Reagan and from George Bush.

It seemed possible that committee members might be willing to forgo examining Mr. Perot's more controversial actions in order to secure his testimony on the central issue of missing Americans and his efforts to find them.

At the heart of these negotiations and Senate deliberations over using its subpoena authority are political worries about how to handle Mr. Perot. On one hand, members like Senator John McCain of Arizona, a former prisoner of war, express a strong desire to conclude a nonpolitical investigation into claims that Americans were left behind in Southeast Asia. Mr. Kerry has said that he has assured Mr. Perot that any hearing on missing Americans will not become a circus.

But at the same time, partisans who oppose Mr. Perot's expected independent candidacy for President see an appearance on Capitol Hill by the Texas billionaire as an opportunity to confront him with recent disclosures about his activities that could be politically embarrassing.

Senate leaders appeared to be ambivalent about pressing Mr. Perot to testify and concerned that the hearings could be poltically damaging to the Senate.

Mr. Perot has seized on this point in bolstering his argument to stay away, referring to the Senate Judiciary Committee's stormy confirmation hearings on the nomination of Clarence Thomas to the Supreme Court. "In the current political climate," he wrote to Mr. Kerry on June 16, "it could become a rerun of the Judge Thomas-Anita Hill hearings."

An influential organization of the families of P.O.W.'s and M.I.A.'s is urging Senator Kerry and his colleagues to subpoena Mr. Perot.

"The league strongly objects to Mr. Perot's decision to renege on his commitment to testify before the select committee," said a statement issued by Sue Scott, the chairwoman of the National League of Families of American Prisoners and Missing in Southeast Asia.

Photo: In 1969, Ross Perot, right, talked with Vu Tien, second from right, an official at the North Vietnamese embassy in Vientiane, Laos. Mr. Perot has refused to testify at a Senate hearing on American servicemen missing in Southeast Asia. (Associated Press)