Paperwork and Regulatory Improvements Act of 2004

Paperwork and Regulatory Improvements Act of 2004 - (Sec. 3) Amends the Paperwork Reduction Act to: (1) require the Director of the Office of Management and Budget (OMB), in consultation with the Internal Revenue Service (IRS) and the Office of Tax Policy of the Department of the Treasury and the Office of Advocacy of the Small Business Administration, in minimizing the Federal information collection burden, to conduct a review of the collections of information by the IRS to identify actions that it can take to reduce the information collection burden imposed on small business concerns, consistent with current Federal law requiring the task force on information collection and dissemination to examine the feasibility of requiring Federal agencies to consolidate requirements regarding collections of information with respect to small business concerns within and across such agencies; and (2) include the results of such review in the annual report submitted to Congress for FY 2006 on the major activities respecting Federal information policy.

(Sec. 4) Amends the Farm Security and Rural Investment Act of 2002 to repeal specified exemptions from Paperwork Reduction Act requirements and certain other rulemaking requirements. Makes the repeal of such exemptions effective 180 days after the enactment of this Act.

(Sec. 5) Amends the Truth in Regulating Act of 2000 to make permanent the authority of a chairman or ranking member of a congressional committee to request the Comptroller General to perform a regulatory analysis of an economically significant rule upon agency publication.

(Sec. 6) Amends the Treasury and General Government Appropriations Act, 2001 to instruct the Director of OMB, in carrying out requirements for the accounting statement and associated report submitted to Congress with the Federal budget submitted to the President, to require each agency annually to submit to OMB an estimate of the total annual costs and benefits of Federal rules and paperwork to the extent feasible for the agency and each agency program.

Requires the Director, after consultation with the head of each agency, to designate at least three agencies (or offices within an agency) to participate in a study on regulatory budgeting for FY 2006 and 2007, including three regulatory agencies or offices from among the Department of Labor, the Department of Transportation, the Department of Health and Human Services, and the Environmental Protection Agency. Requires such study to address the preparation of regulatory budgets and such budgets to include the presentation of the varying estimated levels of benefits that would be associated with the different estimated levels of costs with respect to the regulatory alternatives under consideration by the agency (or office within the agency).

Includes in the accounting statement and associated report submitted to Congress for calendar year 2006 a presentation of the different levels of estimated regulatory benefits and costs with respect to the regulatory alternatives under consideration for at least one of the major regulatory programs of each of the designated agencies.

Requires the Director, after consultation with the Committees on the Budget and on Government Reform of the House of Representatives and the Committees on the Budget and on Governmental Affairs of the Senate, to include in such statement and report submitted to Congress for calendar year 2009 a report on the study which: (1) assesses the feasibility and advisability of including a regulatory budget as part of the Federal budget submitted to the President; (2) describes any difficulties encountered by OMB and the participating agencies in conducting such study; and (3) recommends, to the extent the President considers necessary or expedient, proposed legislation regarding regulatory budgets. Requires such report to also be submitted directly to such congressional committees.