databricks

MARCH 27, 2024

Comments

145

145

Engineering at Meta

MARCH 12, 2024

Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training. We are strongly committed to open compute and open source.

Infiniband 145

Infiniband Data Centers Server Networking 145

Confluent

MARCH 19, 2024

Seamlessly integrate Apache Kafka data into your lakehouse as Apache Iceberg tables, bridging the operational and analytical divide, with Tableflow. Read more in our blog post.

133

133

Dataversity

MARCH 29, 2024

Instead of celebrating World Backup Day 2024 for accomplishing another year of successful backups, I recommend using it to look forward to a year of testing recovery. Instead of starting data protection strategies by planning backups, organizations should flip their mindset and start by planning recovery: What data needs to be recovered first? What systems […] The post World Backup Day Is So 2023 – How About World Data Resilience Day?

Government 119

Government Education 119

Cloudera Blog

MARCH 22, 2024

As enterprise AI technologies rapidly reshape our digital environment, the foundation of your cloud infrastructure is more critical than ever. That’s why Cloudera and Red Hat , renowned for their open-source solutions, have teamed up to bring Red Hat Enterprise Linux ( RHEL ) to Cloudera on public cloud as the operating system for all of our public cloud platform images.

Cloud 115

Cloud Virtual machine DevOps 115

Arista Networks

MARCH 25, 2024

As we all recover from NVIDIA’s exhilarating GTC 2024 in San Jose last week, AI state-of-the-art news seems fast and furious. Nvidia’s latest Blackwell GPU announcement and Meta’s blog validating Ethernet for their pair of clusters with 24,000 GPUs to train on their Llama 3 large language model (LLM) made the headlines. Networking has come a long way, accelerating pervasive compute, storage, and AI workloads for the next era of AI.

Networking 108

Networking Network Ethernet Cloud 108

databricks

MARCH 27, 2024

Databricks’ mission is to deliver data intelligence to every enterprise by allowing organizations to understand and use their unique data to build their.

144

144

Confluent

MARCH 21, 2024

Learn the best practices for using Confluent Schema Registry, including using schema IDs, understanding subjects and versions, using data contracts, pre-registering schemas, and more.

111

111

Dataversity

MARCH 22, 2024

With the recent beta release of Apache Cassandra 5.0, now is a great time for teams to give it a spin and discover 5.0’s most interesting and anticipated new capabilities. As I’ve poked around with the new beta, here are four features introduced with open-source Cassandra 5.0 that developer teams should be excited about: 1. Vector […] The post Four New Apache Cassandra 5.0 Features to Be Excited About appeared first on DATAVERSITY.

Education 115

Education 115

Cloudera Blog

MARCH 6, 2024

As we enter into a new month, the Cloudera team is getting ready to head off to the Gartner Data & Analytics Summit in Orlando, Florida for one of the most important events of the year for Chief Data Analytics Officers (CDAOs) and the field of data and analytics. We’re at a crucial point in time where the excitement and potential surrounding AI has elevated the importance of improving access to the mission-critical data that helps organizations implement it at scale.

Engineering 109

Engineering 109

TDAN

MARCH 20, 2024

Organizations often struggle with finding nuggets of information buried within their data to achieve their business goals. Technology sometimes comes along to offer some interesting solutions that can bridge that gap for teams that practice good data management hygiene.

Artificial Intelligence 105

Artificial Intelligence Government Education 105

databricks

MARCH 19, 2024

Today, we are thrilled to announce that Lilac is joining Databricks. Lilac is a scalable, user-friendly tool for data scientists to search, cluster.

142

142

Engineering at Meta

MARCH 6, 2024

To comply with a new EU law, the Digital Markets Act (DMA), which comes into force on March 7th, we’ve made major changes to WhatsApp and Messenger to enable interoperability with third-party messaging services. We’re sharing how we enabled third-party interoperability (interop) while maintaining end-to-end encryption (E2EE) and other privacy guarantees in our services as far as possible.

Protocol 135

Protocol Encryption Server Media 135

InfoQ Articles

MARCH 21, 2024

Explore Cloudflare's distributed PostgreSQL clusters and learn how a cross-region architecture ensures resilience. Discover how data storage and access at the edge deliver massive performance gains by reducing location-sensitive latency and why architecting for degraded states is much harder than for failure states.

Engineering 105

Engineering 105

Dataversity

MARCH 25, 2024

As shared in part one of this installment, the global marketplace faces an increasingly destructive cyber risk landscape each year, and 2024 is set to confirm this trend. The cost of data breaches alone is expected to reach $5 trillion, a growth of 11% from 2023. As technology advances, attackers continue to develop new, more sophisticated methods […] The post Future-Proof Your Cyber Risk Management with These Top Trends in 2024 (Part II) appeared first on DATAVERSITY.

Education 109

Education 109

DoorDash Engineering

MARCH 27, 2024

Real-time event processing is a critical component of a distributed system’s scalability. At DoorDash, we rely on message queue systems based on Kafka to handle billions of real-time events. One of the challenges we face, however, is how to properly validate the system before going live. Traditionally, an isolated environment such as staging is used to validate new features.

Application 103

Application Engineering Advertising Server 103

TDAN

MARCH 6, 2024

I read “How Big Things Get Done” when it first came out about six months ago.[1] I liked it then. But recently, I read another review of it, and another coin dropped.

Education 105

Education 105

databricks

MARCH 26, 2024

By Steve Sobel - Global Industry Leader; Communications, Media & Entertainment Today Databricks and Adobe are excited to announce a strategic partnership focused.

Media 135

Media 135

Engineering at Meta

MARCH 26, 2024

Meta’s family of apps serves trillions of image download requests every day. And if you’re into high-quality images, you’ve probably noticed that Instagram and Threads have added support for high dynamic range (HDR) photos. Now people on Threads and Instagram can upload and share images that are more true-to-life, with the full color and range their device is capable of capturing.

Media 105

Media Engineering 105

InfoQ Articles

MARCH 6, 2024

The rising popularity of managed relational databases brings hidden costs and informed decisions are crucial for optimal use. This article shows the importance of monitoring service expenses, revising default settings, and understanding operational constraints, considering limitations like reduced flexibility and observability.

Engineering 104

Engineering Cloud 104

Dataversity

MARCH 4, 2024

Hello! I’m Mark Horseman, and welcome to The Cool Kids Corner. This is my monthly check-in to share with you the people and ideas I encounter as a data evangelist with DATAVERSITY. (Last month, we discussed the importance of communication.) This month, we’re talking about neurodiversity in your data team. The term neurodiversity was first […] The post The Cool Kids Corner: Neurodiversity in Your Team appeared first on DATAVERSITY.

Government 105

Government Education 105

Ben Morris

MARCH 28, 2024

Knowing what decisions to make and when to make them can be something of a fine art

100

100

Cloudera Blog

MARCH 28, 2024

At a time when AI is exploding in popularity and finding its way into nearly every facet of business operations, data has arguably never been more valuable. More recently, that value has been made clear by the emergence of AI-powered technologies like generative AI (GenAI) and the use of Large Language Models (LLMs). But, even with the backdrop of an AI-dominated future, many organizations still find themselves struggling with everything from managing data volumes and complexity to security conc

Government 94

Government Cloud 94

databricks

MARCH 13, 2024

Sharing a belief that open source solutions will foster innovation and transparency in generative AI development, Databricks has announced a partnership and participation.

133

133

Engineering at Meta

MARCH 18, 2024

Systems and application logs play a key role in operations, observability, and debugging workflows at Meta. Logarithm is a hosted, serverless, multitenant service, used only internally at Meta, that consumes and indexes these logs and provides an interactive query interface to retrieve and view logs. In this post, we present the design behind Logarithm, and show how it powers AI training debugging use cases.

Engineering 105

Engineering Application Bandwidth Network 105

InfoQ Articles

MARCH 14, 2024

Mike Danilov, a senior principal engineer at AWS, presented on AWS Lambda and what is under the hood during QCon San Francisco 2023. This article represents the talk, which will start with an introduction to Lambda itself to outline the key concepts of the service and its fundamentals, which will facilitate a deep dive into the understanding of the system.

Engineering 104

Engineering DevOps Cloud 104

Dataversity

MARCH 5, 2024

In the realm of database management, particularly with Microsoft SQL Server, understanding and optimizing complex queries is crucial for maintaining system performance and efficiency. As databases grow and complexity, the queries used to retrieve, update, or manipulate data can become increasingly intricate, potentially leading to slower response times and decreased application performance.

Server 105

Server Application Education 105

Confluent

MARCH 19, 2024

Confluent Cloud's serverless Flink offering is now available on all major clouds, offering a unified, managed platform for real-time data processing.

Cloud 93

Cloud 93

Arista Networks

MARCH 5, 2024

Welcome to the digital age, where the marvels of self-driving cars and sophisticated AI like ChatGPT grace our everyday lives. Yet, amidst these advancements, a battleground often goes unnoticed, hidden within the layers of our network infrastructures. It's a world where network teams are the unsung heroes, tirelessly working behind the scenes to keep our digital lifelines seamless and uninterrupted.

Networking 91

Networking Network 91

databricks

MARCH 13, 2024

Introduction Let’s explore a common scenario – your team is eager to leverage open source LLMs to build chatbots for customer support interactions.

119

119

Engineering at Meta

MARCH 20, 2024

At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp. We’ve seen significant benefits by adopting the AV1 codec for RTC. Here’s how we are improving the RTC video quality for our apps with tools like the AV1 codec, the challenges we face, and how we mitigate those challenges.

Bandwidth 104

Bandwidth Networking Network Media 104

InfoQ Articles

MARCH 12, 2024

In this article, we show how you can write advanced macros to step through Rust code and modify it. Using the standard tooling available in the syn crate, we first show how to change the occurrence of a panic into an Err. Then we go a step beyond and use the Fold trait to recursively step through the entire function, automatically executing a change in every applicable location.

Application 97

Application 97

DoorDash Engineering

MARCH 12, 2024

The DoorDash ETA team is committed to providing an accurate and reliable estimated time of arrival (ETA) as a cornerstone DoorDash consumer experience. We want to ensure that every customer can trust our ETAs, ensuring a high-quality experience in which their food arrives on time every time. With more than 2 billion orders annually, our dynamic engineering challenge is to improve and maintain accuracy at scale while managing a variety of conditions within diverse delivery and merchant scenarios.

Engineering 89

Engineering Network Networking 89

Cloudera Blog

MARCH 5, 2024

Artificial Intelligence (AI) is primed to reshape the way just about every business operates. Cloudera research projected that more than one third (36%) of organizations in the U.S. are in the early stages of exploring the potential for AI implementation. But even with its rise, AI is still a struggle for some enterprises. AI, and any analytics for that matter, are only as good as the data upon which they are based.

Data Centers 87

Data Centers Artificial Intelligence Government Cloud 87

Confluent

MARCH 12, 2024

Leverage Confluent Data Streaming Platform to bring real time to your smart warehouse, powering greater IoT automation, efficiency and cost savings.

IoT 84

IoT 84

databricks

MARCH 21, 2024

We are excited to announce the release of the Databricks AI Security Framework (DASF) version 1.0 whitepaper! The framework is designed to improve.

116

116

Dataversity

MARCH 13, 2024

The global market for artificial intelligence (AI) in insurance is predicted to reach nearly $80 billion by 2032, according to Precedence Research. This growth is being driven by the increased adoption of AI within insurance companies, enhancing their operational efficiency, risk management, and customer engagement. Despite widespread integration of AI in the industry today, its full […] The post The Rise of Generative AI in Insurance appeared first on DATAVERSITY.

Insurance 78

Insurance Artificial Intelligence Government Education 78