Engineering at Meta

DECEMBER 6, 2023

We are beginning to upgrade people’s personal conversations on Messenger to use end-to-end encryption (E2EE) by default Meta is publishing two technical white papers on end-to-end encryption: Our Messenger end-to-end encryption whitepaper describes the core cryptographic protocol for transmitting messages between clients. The Labyrinth encrypted storage protocol whitepaper explains our protocol for end-to-end encrypting stored messaging history between devices on a user’s account.

Encryption 145

Encryption Protocol Server Application 145

InfoQ Articles

DECEMBER 13, 2023

Knowing about potential software architecture pitfalls can help teams by keeping them away from tempting paths that will not take them where they want to go. This article covers some of those pitfalls, and provides guidance on how they can be avoided.

138

138

databricks

DECEMBER 7, 2023

Following the announcement we made yesterday around Retrieval Augmented Generation (RAG), today, we’re excited to announce the public preview of Databricks Vector Search. W.

Engineering 132

Engineering 132

Faris Malaeb

DECEMBER 21, 2023

When migrating from Exchange on-premises to Exchange online, you might notice that the mailbox migrations is completed successfully with no failure or error in the migration process. But from Microsoft 365 Admin Portal you find an error about Exchange online archive issue, even though everything seems to be OK for the user in the Exchange. The post FIX: Exchange Failed to enable the new cloud archive using PowerShell appeared first on Faris Malaeb.

Cloud 130

Cloud 130

Confluent

DECEMBER 12, 2023

Dive into the serverless architecture of Confluent Cloud for Apache Flink and explore its benefits like reduced infrastructure costs, increased reliability, & seamless adoption.

Cloud 126

Cloud 126

Dataversity

DECEMBER 18, 2023

Analytics at the core is using data to derive insights for measuring and improving business performance [1]. To enable effective management, governance, and utilization of data and analytics, an increasing number of enterprises today are looking at deploying the data catalog, semantic layer, and data warehouse. But what exactly are these data and analytics tools […] The post Data Catalog, Semantic Layer, Data Warehouse: The Three Key Pillars of Enterprise Analytics appeared first on DATAVE

Government 119

Government Education 119

Engineering at Meta

DECEMBER 19, 2023

On July 5, 2023, Meta launched Threads, the newest product in our family of apps, to an unprecedented success that saw it garner over 100 million sign ups in its first five days. A small, nimble team of engineers built Threads over the course of only five months of technical work. While the app’s production launch had been under consideration for some time, the business finally made the decision and informed the infrastructure teams to prepare for its launch with only two days’ advance notice.

Engineering 144

Engineering Server Data Centers Protocol 144

databricks

DECEMBER 21, 2023

We are excited to announce that Gartner has recognized Databricks as a Leader for a third consecutive year in the 2023 Gartner® Magic.

Cloud 131

Cloud 131

Ben Morris

DECEMBER 12, 2023

The demise of software engineers is one of those tired old tropes that refuses to die. Perhaps the programming revolution has already happened…?

Engineering 100

Engineering 100

Confluent

DECEMBER 12, 2023

Telcos use Confluent with event-driven microservices to enable real-time communications with 3rd-party field service providers, fulfilling customer service requests more efficiently.

110

110

Dataversity

DECEMBER 22, 2023

It sometimes seems like generative AI (GenAI) is sweeping the business world. Not so. We remain in the hype cycle, with forward-thinking organizations refraining from full-blown rollouts for the foreseeable future. Caution is important; organizations that rapidly deploy generative AI integrations have already faced blowback in the form of lawsuits and investor scrutiny.

Education 111

Education 111

Engineering at Meta

DECEMBER 19, 2023

HawkEye is the powerful toolkit used internally at Meta for monitoring, observability, and debuggability of the end-to-end machine learning (ML) workflow that powers ML-based products. HawkEye supports recommendation and ranking models across several products at Meta. Over the past two years, it has facilitated order of magnitude improvements in the time spent debugging production issues.

Engineering 121

Engineering Application 121

InfoQ Articles

DECEMBER 20, 2023

Better documentation and knowledge sharing creates transparency that aids onboarding, prevents turnover disruption, and withstands reorganizations. Different practices can help, such as communicating asynchronously, creating incentives for documentation, making docs discoverable, understanding team members' preferences, and providing dedicated writing time.

100

100

databricks

DECEMBER 6, 2023

Retrieval-Augmented-Generation (RAG) has quickly emerged as a powerful way to incorporate proprietary, real-time data into Large Language Model (LLM) applications. Today we are.

Application 131

Application Engineering 131

DoorDash Engineering

DECEMBER 12, 2023

The DoorDash retail shopping experience mission seeks to combine the best parts of in-person shopping with the power of personalization. While shopping in a physical store has its advantages, a brick-and-mortar store cannot be personalized – the onus is on the consumer to navigate aisles to find what they need. Conversely, a digital shopping experience can be highly personalized.

Real Estate 93

Real Estate Engineering 93

TDAN

DECEMBER 5, 2023

Determining the optimal administrative placement for a data governance program, and specifically a Non-Invasive Data Governance program, is a pivotal decision that can significantly influence its success. This critical choice often boils down to three primary areas within an organization: a specific business area, information technology (IT), or a shared services part of the organization.

Government 83

Government Information Technology Education 83

Dataversity

DECEMBER 11, 2023

Organizations learned a valuable lesson in 2023: It isn’t sufficient to rely on securing data once it has landed in a cloud data warehouse or analytical store. As a result, data owners are highly motivated to explore technologies in 2024 that can protect data from the moment it begins its journey in the source systems. From […] The post Trends in Data Governance and Security: What to Prepare for in 2024 appeared first on DATAVERSITY.

Government 111

Government Cloud Education 111

High Scalability

DECEMBER 11, 2023

Hi everyone, First, I’d like to sincerely thank everyone who has supported Highscalability over the years. We’ve grown together through some interesting times. Massive changes have occurred in how systems are built, and my goal has always been to help people learn how to build them better. I’d like to think that goal was accomplished.

82

82

InfoQ Articles

DECEMBER 29, 2023

Staff engineers can influence behaviors during and after incidents by modeling transparency and questioning assumptions to strengthen engineering culture. As incident commanders, they can coordinate workstreams, communicate with stakeholders, and prevent responder burnout. In retrospectives, staff engineers can improved model root cause analysis to improve underlying cultural issues.

Engineering 96

Engineering 96

databricks

DECEMBER 8, 2023

Retrieval Augmented Generation (RAG) is an efficient mechanism to provide relevant data as context in Gen AI applications. Most RAG applications typically use.

Application 130

Application 130

DoorDash Engineering

DECEMBER 5, 2023

DoorDash’s Engineering teams revamped Kafka Topic creation by replacing a Terraform/Atlantis based approach with an in-house API, Infra Service. This has reduced real-time pipeline onboarding time by 95% and saved countless developer hours. DoorDash’s Real-Time Streaming Platform, or RTSP, team is under the Data Platform organization and manages over 2,500 Kafka Topics across five clusters.

Engineering 91

Engineering Gateway Email Cloud 91

Confluent

DECEMBER 13, 2023

Discover how McAfee saved significant hosting costs alone by shifting to microservices! McAfee’s Mahesh Tyagarajan spills the beans on getting business buy-in and what it means for customers.

80

80

Dataversity

DECEMBER 4, 2023

Hello! I’m Mark Horseman, and welcome to The Cool Kids Corner. This is my monthly check-in to share with you the people and ideas I encounter as a data evangelist with DATAVERSITY. (Read last month’s column here.) This month, we’re talking about data ethics. The technology shift since the global pandemic has given rise to a […] The post The Cool Kids Corner: Current Considerations in Data Ethics appeared first on DATAVERSITY.

Government 105

Government Education 105

Mixpanel

DECEMBER 15, 2023

It’s no secret that the #1 reason why investments in product analytics fail is because the data is untrustworthy. Someone can build the perfect dashboard, but if they can’t answer the question, “Is this data accurate?” nothing matters after that. Today, teams are increasingly interested in pulling trusted data from their warehouse into their analytics tool, which is what we enabled with Warehouse Sync earlier this year.

Engineering 59

Engineering 59

InfoQ Articles

DECEMBER 11, 2023

InfoQ spoke to several technical leaders to understand how they adapt their communication strategies for the current challenges in the distributed and technology-driven workplaces.

82

82

databricks

DECEMBER 21, 2023

Today, Databricks is excited to announce support for Mixtral 8x7B in Model Serving. Mixtral 8x7B is a sparse Mixture of Experts (MoE) open.

Engineering 120

Engineering 120

DoorDash Engineering

DECEMBER 5, 2023

Many organizations use infrastructure-as-code (IaC) with pull request (PR) automation to provide a more secure, safe environment for making infrastructure changes. Despite the power and flexibility of IaC software, the lack of strong, secure defaults in PR automation software can make that sense of security a false one. Infrastructure-as-code and pull request automation IaC enables a declarative, reusable, and auditable way to manage configuration changes.

Engineering 83

Engineering Server Networking Network 83

Confluent

DECEMBER 5, 2023

Confluent Cloud enables organizations to unlock real-time visibility into manufacturing processes, using real-time data collection and analytics to prevent re-work and tooling failures, delivering an outsized impact on production volume and quality.

Cloud 75

Cloud 75

Dataversity

DECEMBER 29, 2023

When it comes to cybersecurity, some enterprises still take it with a grain of salt. But make no mistake about it: No one is safe. Companies such as BBC made it into the list of victims of the most significant data breaches in 2023, contributing to a staggering 5.3 billion total breached records. So, it’s safe […] The post 7 Data Security Best Practices for Your Enterprise appeared first on DATAVERSITY.

Encryption 103

Encryption Education 103

TDAN

DECEMBER 5, 2023

I recently taught an online class on BCBS 239: Effective Risk Data Aggregation and Reporting for Risk.net. Preparing the course materials took me back to 2007-2008, when I worked for Merrill Lynch managing the Credit Risk Reporting team.

Banking 59

Banking Government Education 59

InfoQ Articles

DECEMBER 5, 2023

As an employee or leader, you are in a relationship with each other and with the organization. You face the tension of traditional ways of working vs new ways of working in the call for more work-from-home policies, and more work/life balance. This article outlines the mindset and actions that can be taken to better navigate complex relationships leading to a more humanistic way of working.

65

65

databricks

DECEMBER 12, 2023

Introduction Databricks Lakehouse Monitoring allows you to monitor all your data pipelines – from data to features to ML models – without additional too.

Engineering 117

Engineering 117

Packet Pushers

DECEMBER 27, 2023

Today’s show explores platform engineering. Guest Chad McElligott has thought a lot about the practice of platform engineering, and a blog post and talk he gave about the subject inspired us to reach out and have a conversation. Chad describes platform engineering as “the application of a Product Mindset to supporting your engineering organization’s software.

Engineering 52

Engineering Application 52

Confluent

DECEMBER 4, 2023

Learn how to build and launch streaming apps faster with Data Portal, the developer-friendly, self-service UI to find, understand, and enrich all of your data streams.

Government 69

Government 69

Dataversity

DECEMBER 11, 2023

In an era defined by a surge in technological advancements, artificial intelligence (AI) stands out as a transformative force with the potential to reshape industries, enhance human capabilities, and solve complex problems. However, as AI becomes increasingly integrated into our lives, it’s essential to develop strategies that ensure its long-term viability and ethical use.

Artificial Intelligence 91

Artificial Intelligence Education 91

Obkio

DECEMBER 27, 2023

Immerse yourself in a comprehensive exploration of the top 25 end-user experience monitoring tools. Gain valuable insights into enhancing user satisfaction.

52

52

InfoQ Articles

DECEMBER 26, 2023

Transitioning from a "Compliancе-First" approach to a "Risk-First" mindset rеcognizеs that compliancе should not be viеwеd in isolation but as a componеnt of a broadеr risk managеmеnt strategy.

Application 61

Application DevOps Cloud 61