databricks

OCTOBER 12, 2023

In this blog post, the MosaicML engineering team shares best practices for how to capitalize on popular open source large language models (LLMs).

Engineering 159

Engineering 159

Engineering at Meta

OCTOBER 24, 2023

Meta’s Systematic Code and Asset Removal Framework (SCARF) has a subsystem for identifying and removing dead code. SCARF combines static and dynamic analysis of programs to detect dead code from both a business and programming language perspective. SCARF automatically creates change requests that delete the dead code identified from the program analysis, minimizing developer costs.

Engineering 141

Engineering JavaScript Application Server 141

Confluent

OCTOBER 26, 2023

Read updates and improvements in Apache Flink 1.18, including dynamic fine-grained rescaling via REST API, Java 17 support, and faster rescaling & batch performance improvements.

125

125

DoorDash Engineering

OCTOBER 19, 2023

As DoorDash’s microservices architecture has grown, so too has the volume of interservice traffic. Each team manages their own data and exposes access through gRPC services, an open-source remote procedure call framework used to build scalable APIs. Most business logic is I/O-bound because of calls to downstream services. Caching has long been a go-to strategy to improve performance and reduce costs.

Virtual machine 121

Virtual machine Engineering Bandwidth Application 121

TDAN

OCTOBER 3, 2023

As a frequent reviewer of data and strategy books, I am always interested in understanding authors’ perspectives on data governance. Two recent books have ideas that are worthy of data governance professionals: “Rewired” by Eric Lamarre, Kate Smaje, and Rodney W. Zemmel; and “Data Is Everybody’s Business” by Barbara H. Wixom, Cynthia M.

Government 116

Government Education 116

Dataversity

OCTOBER 2, 2023

Hello! I’m Mark Horseman and welcome to The Cool Kids Corner. This is my monthly check-in to share with you the people and ideas I encounter as a data evangelist with DATAVERSITY. (Read last month’s column here.) This month we’re talking about the current demand for master data management (MDM). What is MDM? Why is it needed now […] The post The Cool Kids Corner: Master Data Management (MDM) in the Spotlight appeared first on DATAVERSITY.

Education 116

Education 116

databricks

OCTOBER 30, 2023

Introduction Four months ago, we shared how AMD had emerged as a capable platform for generative AI and demonstrated how to easily and.

Engineering 142

Engineering 142

InfoQ Articles

OCTOBER 25, 2023

Architectural Decision Records (ADRs) are important vehicles for communicating the architectural decisions a development team makes about a system. Lacking a clear definition of what is architectural, and also lacking anywhere else to record important decisions, they can start to drift from their original purpose and lose focus and effectiveness.

101

101

Mixpanel

OCTOBER 19, 2023

When most people think about data cultures, they imagine multi-screen dashboards with a dizzying array of charts and numbers. But this isn’t what a successful data culture looks like in practice. Simply using data to inform decisions won’t make you succeed at building a product or business any more than buying expensive ingredients will make you a world-class chef.

B2B 98

B2B Engineering 98

Confluent

OCTOBER 12, 2023

With Confluent Cloud, Loggi migrated to an event-driven architecture, powering real-time analytics, boosting productivity, and cutting costs.

Cloud 98

Cloud 98

Dataversity

OCTOBER 24, 2023

In the current economic environment, employee productivity, efficiency, and well-being have become even more critical to organizational success, mandating that leaders spend more time understanding and deriving insights from employees’ digital footprints and data. But too often, businesses make strategic decisions without factoring in workforce data.

Education 115

Education 115

databricks

OCTOBER 31, 2023

Today we're excited to announce MLflow 2.8 supports our LLM-as-a-judge metrics which can help save time and costs while providing an approximation of.

Application 133

Application Engineering 133

Engineering at Meta

OCTOBER 23, 2023

Meta has a very large monorepo, with many different programming languages. To optimize build and performance, we developed our own build system called Buck , which was first open-sourced in 2013. Buck2 is the recently open-sourced successor. In our internal tests at Meta, we observed that Buck2 completed builds approximately 2x as fast as Buck1. Below are five interesting facts you might not have known about Buck2.

Bandwidth 122

Bandwidth Engineering 122

InfoQ Articles

OCTOBER 20, 2023

The JavaScript language and its ever-changing ecosystem of packages and practices can make codebases quickly become hard to maintain. The resulting loss of development velocity and/or code quality can be prevented without rewriting everything from scratch, nor pausing the development of new features. In this article, we have analyzed a few best practices to help avoid that.

JavaScript 100

JavaScript 100

DoorDash Engineering

OCTOBER 24, 2023

At DoorDash, dashing is highly process dependent. Dashers require a firm grasp of the end-to-end delivery process to complete orders successfully — and earn money. The first iteration of DoorDash support content did more to explain how to dash, handle common delivery issues and pitfalls, and maximize the dashing experience than subsequent iterations have done.

Server 83

Server JavaScript Protocol Application 83

Confluent

OCTOBER 11, 2023

Apache Kafka 3.6 brings Tiered Storage Early Access, migrating clusters from ZooKeeper to KRaft with no downtime, a grace period for stream-table joins, and more!

90

90

Dataversity

OCTOBER 9, 2023

Artificial intelligence (AI) is no longer a distant concept – it’s already here, and it’s reshaping the way businesses and society operate. This presents both challenges and opportunities, and as a business leader, your ability to keep up with the intricate world of emerging technologies such as artificial intelligence is crucial. Preparation and execution are […] The post Navigating AI and Its Impact on Business and Society appeared first on DATAVERSITY.

Artificial Intelligence 115

Artificial Intelligence Education 115

databricks

OCTOBER 10, 2023

We’re excited to announce that Meta AI’s Llama 2 foundation chat models are available in the Databricks Marketplace for you to fine-tune and dep.

123

123

Engineering at Meta

OCTOBER 5, 2023

Python 3.12 is out! It includes new features and performance improvements – some contributed by Meta – that we believe will benefit all Python users. We’re sharing details about these new features that we worked closely with the Python community to develop. This week’s release of Python 3.12 marks a milestone in our efforts to make our work developing and scaling Python for Meta’s use cases more accessible to the broader Python community.

Engineering 120

Engineering Server 120

InfoQ Articles

OCTOBER 4, 2023

Jakarta Data streamlines Java enterprise data integration. Supporting various databases, it boosts productivity, is open-source, and community-driven. GitHub offers hands-on experience for modernizing enterprise architectures.

DevOps 89

DevOps Engineering 89

DoorDash Engineering

OCTOBER 31, 2023

At DoorDash, the growth and development of our engineering talent is critical to our success and ability to continue innovating. Apprenticeship has had a long history of successfully cultivating new generations of talent across many different industries. Tech is no different. Designed to identify and foster exceptional engineering talent within the company, DoorDash’s Apprentice Engineering Manager Program prepares engineers to transition into a people management role effectively and autonomousl

Engineering 80

Engineering Education Networking Network 80

Confluent

OCTOBER 9, 2023

Learn how data streaming and artificial intelligence enables you to project your brand’s reputation with real-time social media monitoring.

Artificial Intelligence 80

Artificial Intelligence Media 80

Dataversity

OCTOBER 27, 2023

In today’s digital world, data rules. Yet information must remain confidential to have any value in a business context. Customer data, financial records, and intellectual property are susceptible to cyber threats. As a result, reinforcing security is a must for organizations that want to keep their reputation. This is where data masking comes in. What Is […] The post Data Masking Best Practices and Benefits appeared first on DATAVERSITY.

Financial 115

Financial Government Education 115

databricks

OCTOBER 25, 2023

Predictive Optimization intelligently optimizes your Lakehouse table data layouts for peak performance and cost-efficiency - without you needing to lift a finger.

112

112

Engineering at Meta

OCTOBER 18, 2023

With the recent launches of MTIA v1 , Meta’s first-generation AI inference accelerator, and Llama 2 , the next generation of Meta’s publicly available large language model, it’s clear that Meta is focused on advancing AI for a more connected world. Fueling the success of these products are world-class infrastructure teams, including Meta’s custom AI silicon team, led by Olivia Wu, a leader in the silicon industry for 30 years.

Engineering 115

Engineering Data Centers Application Media 115

InfoQ Articles

OCTOBER 27, 2023

Struggling with your "agile transformation?" Is your scaling framework not providing the outcomes you hoped for? In this article, we’ll discuss how teams in a large enterprise replaced heavy agile processes with Conway’s Law and better engineering to migrate from quarterly to daily value delivery to the end users.

Engineering 83

Engineering 83

TDAN

OCTOBER 31, 2023

The concept of “walking the data factory” drew a great deal of interest during our recent DGPO webinar on data classification as part of a holistic governance program. We discussed ways to connect the stove-piped worlds of data governance and information governance under a common governance classification.

Government 59

Government Education 59

Confluent

OCTOBER 25, 2023

Debugging Apache Kafka® issues shouldn’t send shivers down your spine. Explore the latest blog posts, on-demand videos, and demos on Kafka troubleshooting to ease your fears.

78

78

Dataversity

OCTOBER 30, 2023

In today’s rapidly evolving business landscape, decision-making has shifted from relying on gut instinct and the loudest voice in the room to data-driven approaches. In this article, we will explore five powerful ways to harness data for making informed decisions in the workplace, offer practical insights to boost productivity, and explain how to adapt to […] The post 5 Ways to Use Data to Make Workplace Decisions appeared first on DATAVERSITY.

Education 111

Education 111

databricks

OCTOBER 19, 2023

Machine learning (ML) is more than just developing models; it's about bringing them to life in real-world, production systems. But transitioning from prototype.

108

108

Engineering at Meta

OCTOBER 31, 2023

Meta’s Systematic Code and Asset Removal Framework (SCARF) has a subsystem for identifying and removing unused data types. SCARF scans production data systems to identify tables or assets that are unused and safely removes them. SCARF avoids tedious manual work and ensures that product data is correctly removed when a product is shut down. This is the third and final post in our series on Meta’s Systematic Code and Asset Removal Framework (SCARF).

Engineering 107

Engineering Education Server 107

InfoQ Articles

OCTOBER 13, 2023

In this article, we will present our approach for analysing agile systems as networks of queues and how we have used it to analyse 926 projects in the Public Jira Dataset. We explain how you can measure the Stability Metric (SM) for your queues. Finally, we will present our planned next phase of research.

Network 61

Network Networking 61

TDAN

OCTOBER 17, 2023

According to the authors of “Data Is Everybody’s Business,” a data strategy “lays out an organization’s goals and plans for managing and exploiting data.” So, where are chief information officers (CIOs) at in facilitating a data strategy with their business counterparts? What things get in the way the most?

Government 59

Government Education 59

Confluent

OCTOBER 11, 2023

Discover data-centric security strategies with Confluent. Join Mike Peacock on Oct 12 for key insights. Register now!

78

78

Dataversity

OCTOBER 3, 2023

While studies show that women currently hold about 28% of technology jobs, the total number of women in tech positions has actually decreased over the last few years. Gender disparity clearly still exists, and it is even more prominent in the field of Data Science. To make way for more women in Data Science, it’s important for me to share […] The post Lessons Learned from a Woman in Data Science appeared first on DATAVERSITY.

Education 111

Education 111

databricks

OCTOBER 24, 2023

"I can't think of anything that's been more powerful since the desktop computer." — Michael Carbin, Associate Professor, MIT, and Founding Advisor, MosaicML A.

103

103

Mixpanel

OCTOBER 13, 2023

Last year, we covered the importance of polishing your product’s workflows to refine the value you’ve built for your users. The key message was that incremental features and improvements aren’t meant to create a lot of net new value but instead remove friction from a current product experience and build upon existing value. Polishing existing workflows is still one of our strongest priorities at Mixpanel, and we take careful consideration into how we can reshape our product to better serve your

59

59