开源AI

This study evaluates the performance of general-purpose AI, like ChatGPT, inlegal question-answering tasks, highlighting significant risks to legalprofessionals and clients. It suggests leveraging foundational models enhancedby domain-specific knowledge to overcome these issues. The paper advocates forcreating open-source legal AI systems to improve accuracy, transparency, andnarrative diversity, addressing general AI's shortcomings in legal contexts.

Abstract Healthcare datasets are becoming larger and more complex, necessitating the development of accurate and generalizable AI models for medical applications. Unstructured datasets, including medical imaging, electrocardiograms, and natural language data, are gaining attention with advancements in deep convolutional neural networks and large language models. However, estimating the generalizability of these models to new healthcare settings without extensive validation on external data remains challenging. In experiments across 13 datasets including X-rays, CTs, ECGs, clinical discharge summaries, and lung auscultation data, our results demonstrate that model performance is frequently overestimated by up to 20% on average due to shortcut learning of hidden data acquisition biases (DAB). Shortcut learning refers to a phenomenon in which an AI model learns to solve a task based on spurious correlations present in the data as opposed to features directly related to the task itself. We propose an open source, bias-corrected external accuracy estimate, P E s t , that better estimates external accuracy to within 4% on average by measuring and calibrating for DAB-induced shortcut learning.

MortalityMinder enables healthcare researchers, providers, payers, and policy makers to gain actionable insights into where and why premature mortality rates due to all causes, cancer, cardiovascular disease, and deaths of despair rose between 2000 and 2017 for adults aged 25–64. MortalityMinder is designed as an open-source web-based visualization tool that enables interactive analysis and exploration of social, economic, and geographic factors associated with mortality at the county level. We provide case studies to illustrate how MortalityMinder finds interesting relationships between health determinants and deaths of despair. We also demonstrate how GPT-4 can help translate statistical results from MortalityMinder into actionable insights to improve population health. When combined with MortalityMinder results, GPT-4 provides hypotheses on why socio-economic risk factors are associated with mortality, how they might be causal, and what actions could be taken related to the risk factors to improve outcomes with supporting citations. We find that GPT-4 provided plausible and insightful answers about the relationship between social determinants and mortality. Our work is a first step towards enabling public health stakeholders to automatically discover and visualize relationships between social determinants of health and mortality based on available data and explain and transform these into meaningful results using artificial intelligence.