For All Subjects: 1. Age (at time of consent/assent): >= 6 months to <= 21 years of age * Cohort 4 only: >= 10 years to <= 21 years 2. Performance Status: * If < 12 years of age: Lanksy Performance Status > 50% * If >= 12 years of age: Karnofsky Performance Status > 50% NOTE: If subject is unable to walk due to paralysis, but is mobile in a wheelchair, subject is considered to be ambulatory for the purpose of assessing their performance status. 3. Has provided signed written informed consent/assent 4. Has a life expectancy of > 3 months 5. Has relapsed or refractory disease and no standard treatment options as determined by locally or regionally available standards of care and treating physician's discretion 6. Is ineligible or inappropriate for other treatment regimens known to have effective potential 7. Has a documented local diagnostic pathology of original biopsy confirmed by a Clinical Laboratory Improvement Amendments (CLIA)/College of American Pathologists (CAP) or other Sponsor-approved laboratory certification 8. Has all prior treatment (i.e., chemotherapy, immunotherapy, radiotherapy) related clinically significant toxicities resolve to <= Grade 1 per Common Terminology Criteria for Adverse Events (CTCAE), version 4.03 or are clinically stable and not clinically significant, at time of enrollment 9. Prior therapy(ies), if applicable, must be completed according to the criteria below: Prior Therapy & Time from Last Prior Therapy * Other investigational agent (any medicinal product that is not approved in the country for treatment in any indication, adult or pediatric): At least 30 days or five half-lives, whichever is longer, since last dose of an investigational agent prior to the first dose of tazemetostat * Chemotherapy- cytotoxic: At least 21 days since last dose of chemotherapy prior to the first dose of tazemetostat * Chemotherapy - nitrosoureas: At least 6 weeks since last dose of nitrosoureas prior to the first dose of tazemetostat * Chemotherapy - non-cytotoxic (e.g., small molecule inhibitor): At least 14 days since last dose of non-cytotoxic chemotherapy prior to the first dose of tazemetostat * Monoclonal antibody(ies): At least 28 days since last dose of any monoclonal antibody prior to the first dose of tazemetostat * Immunotherapy (e.g., tumor vaccine): At least 6 weeks since last dose of immunotherapy agent(s) prior to the first dose of tazemetostat * Radiation therapy (RT): - At least 14 days from last local site RT prior to first dose of tazemetostat - At least 21 days from stereotactic radiosurgery prior to first dose of tazemetostat - At least 12 weeks from craniospinal, >= 50% radiation of pelvis, or total body irradiation prior to first dose of tazemetostat * Hematopoietic growth factor: At least 14 days from last dose of hematopoietic growth factor prior to the first dose of tazemetostat * Hematopoietic cell transplantation (allogeneic or autologous): At least 60 days from infusion of hematopoietic cells prior to the first dose of tazemetostat 10. Has adequate hematologic (bone marrow and coagulation factors), renal and hepatic function as defined by criteria in the protocol. NOTE: Laboratory results obtained during Screening should be used to determine eligibility criteria. In situations where laboratory results are outside the permitted range, the Investigator may retest the subject and the subsequent within range screening result may be used to determine the subject[Single Quote]s eligibility. 11. For subjects with central nervous system (CNS) involvement: Subject must have: * deficits that are stable for a minimum of 14 days prior to first dose of study drug, or * seizures that are stable, not increasing in frequency or severity and controlled on current anti-seizure medication(s) for a minimum of 7 days prior to first dose of study drug * treated brain metastases without evidence of progression (new or enlarging brain metastases by imaging 4 weeks prior to the first dose of study drug) and are on stable or tapering doses of steroids for at least 7 days prior to first dose of study drug NOTE: Subjects with leptomeningeal disease or brain tumors with positive cerebral spinal fluid cytology are eligible for this study. Subjects may receive glucocorticoids (at stable or tapering dose) to control CNS symptoms prior to enrollment; however, subjects should receive a stable or tapering dose for at least 7 days prior to first dose of study drug. 12. Has a shortening fraction of > 27% or an ejection fraction of >= 50% by echocardiogram (ECHO) or multi-gated acquisition (MUGA) scan and New York Heart Association Class <= 2 13. Has a QT interval corrected by Fridericia's formula (QTcF) <= 450 msec 14. Is able to swallow and retain orally administered medication and does not have any uncontrolled gastrointestinal (GI) condition such as nausea, vomiting or diarrhea, or any clinically significant GI abnormalities that may alter absorption such as malabsorption syndrome, hereditary fructose intolerance, glucose-galactose malabsorption, sucrose-isomaltase insufficiency, or major resection of the stomach and/or bowels. NOTE: Nasogastric and gastrostomy tube administration of the oral suspension formulation of study drug is permitted. 15. Has sufficient tumor tissue (slides or blocks) available for central confirmatory testing of immunohistochemistry (IHC) and/or cytogenetics/fluorescence in situ hybridization (FISH) and/or DNA mutation analysis (required for study entry but enrollment based on local results). 16. Is willing and able to comply with all aspects of the protocol as judged by investigator 17. For female subjects of childbearing potential: Subject must: * Have a negative beta-human chorionic gonadotropin ([BETA]-hCG) pregnancy test at time of Screening and within 72 hours prior to planned first dose of tazemetostat (urine or serum test is acceptable however, positive urine tests must be confirmed with serum testing), and * Agree to use effective contraception, as defined in in the protocol from start of screening until 30 days following the last dose of study treatment and have a male partner who uses a condom, or * Practice total abstinence (when this is in line with the preferred and usual lifestyle of the subject), or * Have a male partner who is vasectomized with confirmed azoospermia 18. For male subjects with a female partner of childbearing potential: Subject must: * Be vasectomized or * Agree to use effective contraception as defined in the protocol from 30 days prior to the first dose of tazemetostat until 30 days following the last dose of tazemetostat, or * Have a female partner who is NOT of childbearing potential 19. For French subjects only: Is either affiliated with or a beneficiary of a social security category For Dose Escalation only To be eligible for enrollment in Dose Escalation, a subject must meet ALL of the following criteria in addition to the inclusion criteria above: 1. Has evaluable disease as defined as lesions that can be accurately measured at least in one dimension by radiographic examination or physical examination or other lesions such as bone lesions, leptomeningeal disease, ascites, pleural/pericardial effusions, lymphangitis cutis/pulmonitis or hepatosplenomegaly from disease. 2. Has one of the following histologically confirmed tumors: * Rhabdoid tumor: * Atypical teratoid rhabdoid tumor (ATRT) * Malignant rhabdoid tumor (MRT) * Rhabdoid tumor of the kidney (RTK) * Selected tumors with rhabdoid features * Integrase interactor 1 (INI1)-negative tumor: * Epithelioid sarcoma (ES) * Epithelioid malignant peripheral nerve sheath tumor * Extraskeletal myxoid chondrosarcoma (EMC) * Myoepithelial carcinoma * Renal medullary carcinoma (RMC) * Other INI1-negative malignant tumors (e.g., dedifferentiated chordoma) NOTE: Requires prior Sponsor approval * Synovial sarcoma with SS18-SSX rearrangement NOTE: Evidence of diagnostic pathology of original biopsy confirmed by a CLIA/CAP or other Sponsor-approved certified laboratory must be available. 3. For subjects with ATRT, MRT, RTK, and selected tumors with rhabdoid features only: The following test results must be available: * Morphology and immunophenotypic panel consistent with rhabdoid tumor, and * Loss of INI1 or SMARCA4 confirmed by immunohistochemistry (IHC), or * Molecular confirmation of tumor bi-allelic INI1or SMARCA4 loss or mutation when INI1 or SMARCA4 IHC is equivocal or unavailable 4. For subjects with INI1-negative tumor only: The following test results must be available: * Morphology and immunophenotypic panel consistent with INI1-negative tumors, and * Loss of INI1 confirmed by IHC, or * Molecular confirmation of tumor bi-allelic INI1 loss or mutation when INI1 IHC is equivocal or unavailable 5. For subjects with synovial sarcoma only: The following test results must be available: * Morphology consistent with synovial sarcoma, and * Cytogenetics or fluorescence in situ hybridization (FISH) and/or molecular confirmation (e.g., deoxyribonucleic acid [DNA] sequencing) of SS18 rearrangement t(X;18)(p11;q11) For Dose Expansion only To be eligible for enrollment in the Dose Expansion part, a subject must meet ALL of the following criteria in addition to the inclusion criteria for All Subjects listed above: 1. Has measurable disease as defined in the protocol 2. Has one of the following histologically confirmed rhabdoid tumors: Cohort 1: * ATRT Cohort 2: * MRT * RTK * Selected tumors with rhabdoid features Cohort 3 (INI1-negative tumors): * Epithelioid sarcoma (ES) * Epithelioid malignant peripheral nerve sheath tumor * Extraskeletal myxoid chondrosarcoma (EMC) * Myoepithelial carcinoma * Renal medullary carcinoma (RMC) * Chordoma (poorly differentiated or de-differentiated) * Other INI1-negative malignant tumors NOTE: Requires prior Sponsor approval Cohort 4 * One of the tumor types eligible for Cohorts 1 through 3 or synovial sarcoma with SS18-SSX rearrangement NOTE: Evidence of diagnostic pathology of original biopsy confirmed by a CLIA/CAP or other Sponsor-approved certified laboratory must be available. 3. For subjects with ATRT/MRT/RTK only - Have the following test results available: * Morphology and immunophenotypic panel consistent with rhabdoid tumor, and * Loss of INI1 or SMARCA4 confirmed by IHC, or * Molecular confirmation of tumor bi-allelic INI1 or SMARCA4 loss or mutation when INI1 or SMARCA4 IHC is equivocal or unavailable 4. For subjects with INI1-negative tumors only x have the following test results available: * Morphology and immunophenotypic panel consistent with INI1-negative tumors, and * Loss of INI1 confirmed by IHC, or * Molecular confirmation of tumor bi-allelic INI1 loss/mutation when INI1 IHC is equivocal or unavailable 5. For subjects with synovial sarcoma with SS18-SSX rearrangement only (Cohort 4) - have the following test results available: * Morphology consistent with synovial sarcoma, and * Cytogenetics or FISH and/or molecular confirmation (e.g., deoxyribonucleic acid [DNA] sequencing) of SS18 rearrangement t(X;18)(p11;q11) 6. For subjects to be enrolled in Cohort 4: Able to swallow and retain orally administered tablets