Fiscally Responsible Family Tax Relief Act of 2004

Fiscally Responsible Family Tax Relief Act of 2004 - Amends the Internal Revenue Code to extend through 2005: (1) the $1,000 (per child) child tax credit amount; (2) the increased standard deduction for married taxpayers; and (3) the increased ceilings for the 15 and ten percent tax brackets for married taxpayers. Eliminates the reduction in the refundability (15 to ten percent) of the child tax credit. Includes combat zone compensation as earned income for purposes of calculating the refundable portion of the child tax credit.

Sets forth a uniform definition of "child" for purposes of applying certain tax provisions, including the dependent care credit, the child tax credit, and the deduction for personal exemption for dependents.

Sets forth rules for the application of the economic substance doctrine. Increases penalties for promoting abusive tax shelter transactions. Proposes new restrictions on tax shelter activities, including increased disclosure requirements, denial of a tax deduction for interest on underpayments attributable to undisclosed tax shelter transactions, and denial of privilege for communications relating to certain tax shelter transactions.

Proposes restrictions on corporate tax shelters, including: (1) a limitation on the transfer or importation by a corporation of built-in losses; (2) the repeal of special rules for Financial Asset Securitization Investment Trusts (FASITS); and (3) expansion of the authority of the Secretary of the Treasury to deny tax benefits related to tax shelter transactions.

Amends the Consolidated Omnibus Budget Reconciliation Act of 1985 (COBRA) to extend the authority for certain customs user fees through September 30, 2013.