Arrested Bengal minister Partha Chatterjee, who is at the centre of a multi-crore school job scam, on Sunday claimed that the money recovered during Enforcement Directorate (ED) raids does not belong to him, and only time will tell who all have "conspired" against him.

As he deboarded a vehicle outside ESI Hospital at Joka, where he was taken for a medical check-up, Chatterjee, on being approached by reporters, said, "The money (recovered) is not mine."

Asked further if anybody was conspiring against him, he said, "You will get to know when the time comes."

Later, on his exit from the hospital, Chatterjee reiterated that the money didn't belong to him, and he "has never been involved in such dealings".

Nearly Rs 50 crore in cash along with gold, value of which is still being ascertained, were recovered from apartments linked to Arpita Mukherjee, one of Chatterjee's close aides, besides documents of properties, as per officials of the ED, the agency that arrested him.

Chatterjee had on Friday asserted that he was a victim of a conspiracy and expressed unhappiness over TMC's decision to suspend him.

"This decision (to suspend me) could influence an impartial probe," he had said.

About the move to remove him from the ministry, Chatterjee, once considered a close confidante of Chief Minister Mamata Banerjee, said, "Her decision is right."

Chatterjee, 69, was relieved of his duties as minister-in-charge of various departments and suspended from the TMC on Thursday. He was also removed from all party posts.

His close aide, Mukherjee, has also been arrested by the ED.

The TMC leadership has taken exception to his comments, contending that Chatterjee is himself responsible for his fate.

"Why was he silent for the last few days after his arrest? He has every right to approach the court and prove his innocence. The party has nothing to do with this scam. We do not wish to comment on his remarks regularly," TMC state general secretary Kunal Ghosh said.

The Opposition, however, went at Chatterjee's comments hammer and tongs, and said he should clarify who is the owner of the recovered money .

"If the money recovered doesn't belong to Partha Chatterjee, then whose is it? He should say this, people of this state want to know who is the owner of this looted money. He should spill the beans," BJP state president Sukanta Majumdar said.

During the day, a rally was taken out by the BJP in Narayangarh area of Paschim Medinipur, demanding the arrest of all TMC thieves .

Today I joined a rally at Khakurda bazar, Narayanagarh to demand the immediate arrest of TMC thieves and the removal of Dacoit Cabinet of Paschim Banga. Today's BJP rally turned into a mass rally. Many common people raised their voices in protest against corruption (sic), BJP national vice president Dilip Ghosh later tweeted.

CPM leader Sujan Chakraborty said Chatterjee should reveal the secrets behind the organised recruitment scam.

"His comments reflect that many top leaders of the TMC are involved in the scam. He should reveal those names and ensure that everyone involved are sent behind bars," Chakraborty asserted.

The CBI, as directed by Calcutta High Court, is probing the alleged irregularities in the recruitment of Group-C and -D staff, as well as teachers in government-sponsored and -aided schools, on the recommendations of the West Bengal School Service Commission.

The ED is looking into the money trail involved in the scam.

Chatterjee had held the education portfolio when the scam was allegedly pulled off.